No Analytics, No Audit Department

dead-audit-department

If YOUR audit department doesn’t embrace data, analytics, and automation eventually, your audit department will NOT exist.

No data, no analytics. No analytics, no automation. Eventually, no audit department.

Editor Note: This post really applies to all departments in a company, but mainly I’m addressing auditors, but you might want to read between the business lines….

By embrace, I don’t mean have one or two auditors working on this. I mean the entire department.

Before you cite all the regulatory requirements mandating the existence of an audit department in companies, having an audit department in name only won’t cut it.

Having an inept audit department will not be acceptable to regulators, and it shouldn’t be acceptable to company management either. Or Audit Committees!

Companies need skilled and efficient auditors that can do the heavy lifting, and this need will only increase.

The (Audit) World is Changing

I’m saying that internal audit is changing, and auditors who don’t want to change with it will be left behind.

Consider, as an example: Some years ago, automobiles had NO on-board computers. They were basically mechanical machines. When computers arrived in automobiles, repair specialists had to learn how to troubleshoot and work with them, or they found another job or retired.

A similar transition is occurring in the business world, but most auditors refuse to learn how to handle and manipulate data, perform analytics, and automate their work. This trend is not a fad, and it isn’t going away. Only old-school auditors are.

Why?

Why do I think some audit departments will be obsolete in a few years?

  1. Audits are becoming more technical because business processes rely more and more on technology.
    And yet all the audit publications and consultancies STILL complain that auditors don’t understand data or analytics; too few audits include deep data analysis; fewer auditors are learning automation.
  2. More business processes generate or consume data.
    The age of paper and tick marks is over. Data tells the story of whether the controls are working, whether fraud may be spawning, or whether a process is inefficient. Hardly any business process doesn’t depend on data.
  3. Business units are doing their own dashboards and analytics.
    How come business-line leaders can see that deep data analysis and dashboards provide better information, reduce risk, and in some cases, save money and time, and yet most audit leaders still don’t get it?
  4. Companies are starting to focus more heavily on automation.
    Automation is becoming more necessary as companies compete; the bar for survival is getting raised. Why do audit departments think they are exempt from this?

Points #1 & 2 above are why auditors are doing more integrated audits–audits that have a financial/operational component AND a technology/IT component.

But overall, audit departments are NOT adapting and growing their skills.

Business Departments are Pulling Ahead

Meanwhile, business departments in a lot of companies:

  1. Have been training their employees in analytics and hiring staff with data and analytics experience.
  2. Are not only doing more and more analytics, but more complicated analytics.
  3. Are automating all data extraction, and are now automating more of their analyses.

In other words, business departments have more expertise in analytics and automation than internal audit, and yet internal audit is not in much of a hurry to catch up.

Previously, I’ve said that individual auditors will die if they don’t do analytics (see my earlier post), so what do you think will happen to an entire department full of non-analytic auditors?

A department’s death will be much slower; it will start with the department becoming less and less relevant.

Maybe your company isn’t becoming data driven yet. But other companies ARE, so what impact do you think those competitors will have on your company? And will that be good for your audit department?

Here’s how it can happen

1.) Auditors don’t know whether the data they receive is the RIGHT data. When I review the queries that IT or the business run to provide data to the audit department, I often find errors that that leave out too many critical records, or include ones I don’t need, or both.

Most auditors, including many IT auditors, don’t know how to read and interpret queries, so they can’t be confident of the data. This is not a new risk, but as more auditors and businesses do analytics, it is more important than ever.

Therefore, I am not surprised at the low confidence levels noted in a recent article at the ACL website:

See this article, which states “17% of internal audit teams have a high degree involvement in evaluating the quality of data used and 47% have little or no involvement.”

If your data is poor, your analytics will be misleading or flat wrong.

Many auditors don’t have enough understanding of query languages and basic network/file/database knowledge to even ask subject matter experts intelligent questions or recognize whether the answers they receive are reasonable.

2.) Auditors who are unfaithful in little are unfaithful in much.

If audit departments are not diligent enough to validate and profile their data, they most likely won’t do the more complicated analyzes either.

Profiling data in Excel or ACL is EASY and FAST. Profiling also helps you determine which products, transactions, or systems to focus your audit on, and what can be deferred or left untested.

If your auditors don’t do the easy stuff, how will they learn to do the heavy lifting for in-depth analysis or automation?

3.) Auditors are not keeping up with the business. More business lines do more and more of their own analytics, but audit departments continue to lag behind.

While audit departments seem to have a good grasp of the business processes, they don’t have a good grasp of the data or analytic processes (or tools) their business partners are beginning to depend on.

4.) Auditors will soon need to know how to audit business line analytic processes and data models. The best way to learn this is to do your own analytics (surprise!) and learn about the tools the business is using. If auditors aren’t doing analytics, CAEs will have to send auditors to a ‘checklist class’ or hire consultants to do that auditing.

Eventually, business lines will be analyzing most of their critical data.

So when you start an audit and ask for data, the business lines are going to tell you they already analyze their data, so why should you repeat what they have already done? They will tell you to just review THEIR analytics.*

*This recently occurred in a process I was auditing. The process isn’t in production yet, but it is running hard in the test environment and producing results. The business manager told me he expects to replace some manual controls with his new analytic process.

And since your department is so far behind the analytic curve, you won’t be able to audit their analytic process.

So you’ll need to hire a third party at great cost. That’s the path to irrelevance. Or  maybe outsourcing.

CAEs need to get ahead of this shift.

CAEs (and Audit Committees!) need to understand auditors don’t need to do more analytics just to speed up their audits, provide more coverage, do more complex audits, and look good to the audit committee–auditors also need to gain a deep understanding of how to audit the analytics and data models their business lines are starting to depend on.

And that means they have to understand a lot more about analytics than what’s needed to run robust ACL analytics (which most companies still don’t do!).

As much as I love ACL, only 1 of my business lines use it (thanks to me).

That means that I also need to understand and use some of the tools the business uses. While ACL is much simpler to use, the tools the business uses are often faster, but more complex.

But as I master the the tools the business use, my influence with the business grows. I don’t have just one wrench (ACL) in my audit toolkit. And I understand the data and the process that creates it better.

Now What?

Some suggestions:

  • Take ownership for consistently increasing your own data, analytic, and automation skills, regardless of what your company or audit department are doing (or NOT).
  • Think about how your department has progressed  in handling data, performing analytics, and enabling automation in the past 2 years. If progress has been minimal, ask why. Then ask what’s the cost and impact of continuing on a similar path.
  • Benchmark your audit management against the concerns raised in 10 Signs Mgmt Doesn’t Really Support Analytics. Create and execute a plan for improvement.
  • If you can’t gain any traction getting analytics going in your department, or progress is slower than it should be, leave this post on your management’s desk*. Anonymously, of course.
  • Start looking for a job in a more progressive industry and/or company.

* Please let me know what happened!

Advertisements

17 Comments

Filed under Audit, Data Analytics, Employment, Technology, Written by Skyyler

17 responses to “No Analytics, No Audit Department

  1. completely agree with you. It’s high time for auditors to embrace the data analytics and automation.

    Liked by 1 person

  2. AuditB

    Great article and I agree. One more thought or “look into the crystal ball” if you will. IT audit as we know it today will be extinct in the near future. Your either an auditor or a security auditor. IT is becoming too specialized for a general auditor to stay relevant and security reviews require a different skill set. With more integrated audits occurring, the need for a separate IT audit is nonsense. Curious on your thoughts or if you see it heading in that direction as well?

    Like

    • AuditB,
      That’s a great question. I’m all over the map on that subject, so I’m not sure what to say. Here are my thoughts.

      By security auditor, I assume you mean someone more skilled than a good IT auditor, who’s as much as a security analyst as an auditor.

      In my experience (and as I’ve said many times on my blog), I don’t think most companies really value skilled IT auditors. My reasons: 1) CISA exam is too easy to pass without really understanding IT, 2) employers hire unskilled IT auditors or have general auditors do IT audits, and 3) a shortage of good IT auditors exists in the US, which I think contributes heavily to 1) and 2) above.

      As a result, I don’t see that most companies value security auditors. I have more experience and expertise than most IT auditors I’ve met, and I also have good data wrangling and data analytic skills. I don’t feel very valued; like everyone else, I am just a commodity to most employers to be bought, used, traded, and discarded. That just seems to be the corporate climate in the U.S., whether you’re a contractor or regular employee. But I digress.

      Several of the companies I have audited struggle to maintain basic SOX controls. I see the same issues coming up again and again, and the tone at the top hasn’t taken hold. Therefore, why would these companies value skilled auditors who could look deeper into security and find even more things that need attention?

      Any yet integrated audits are increasing. But what I am seeing is that there’s still a general auditor piece and and IT piece. While the two auditors talk to each other more and help each other understand how the IT piece could affect financials or efficiency, and vice versa, most of the audits I see are two audits blended into one.

      Or a general auditor is just looking at user access, and they call that an integrated audit, which it isn’t, but it makes audit managers, CAEs, and audit committees feel better about the number of integrated audits that were completed.

      I think the same thing is happening, more than not, with analytics. Many general auditors don’t want to do analytics and leave that to the IT auditors or the analytic auditors.

      [Some general auditors do great analytics, but on the whole they don’t; we know that because companies employ more general auditors than IT or analytic auditors, and everyone from IIA, ISACA, and the Big 4 complain analytics isn’t mainstream yet).

      So just as most general auditors don’t do analytics, most general auditors don’t do real IT and/or security audits. I’m starting to wonder if they think analytics and IT/security is beneath them–their mighty CPA designations are all they need; besides, analytics is just a fad, and IT/security is just too hard to understand, so it must not be important.

      The other thing that makes me think that security auditors won’t be the standard any time soon is the security failures that occur at companies each week.

      If these companies really cared about security, their security would be better. (Keep in mind I was the head of security at a Fortune 500 previously, and while you can’t prevent all security failures, many of those we hear about are the result of exploiting unpatched (often old) vulnerabilities that everyone knew about).

      So if most companies aren’t diligent about keeping out of the newspapers, don’t value their own security and investing in their security team, why would they value security auditors? Or even decent IT auditors?

      Finally, those of us that are decent IT auditors know that the companies we work for often take years to fix security problems, and many of them they just ‘accept’ as the risk of doing business (again, I understand perfect security is impossible and good security is just plain expensive, but we all know too many security issues that CAN be fixed in a reasonable time frame, at a reasonable cost, but are NOT).

      So in summary, I guess I don’t see IT auditors dying out and a new breed of security auditors rising. If anything, I see the demise of the general auditor, because eventually, if you don’t know technology, you won’t be able to audit.

      Any auditor who doesn’t have basic technology skills won’t survive because they won’t have the foundation to constantly learn new technology, understand new risks and how changes in technology affect that risk, and as a result, they will not be able to adapt.

      That’s also why if you don’t embrace analytics (which requires you to understand the ‘old’ technology of files, networks, firewalls, databases, data wrangling, etc.), you won’t be able to adapt to where analytics is taking the audit industry and all industries.

      One last thought that escapes most general auditors: basic financial principles do NOT change; sure, there’s new types of financial accounts, deals, money-making schemes, and tax laws, but overall, the underlying principles are the same.

      However, technology is rapidly evolving. From PCs to mainframes to client server to virtual technology mobile apps to Bitcoin and blockchain, IT is getting more complicated and harder to secure.

      IT and security auditors have so much more to keep up with (any finance folks and CPAs, if you disagree, I’d love to hear your reasons), and without good IT, you can’t have good finance–that’s what SOX is all about. Furthermore, I think it’s harder to have good, solid IT than good financial control over a company.

      Anyway, I’ve rambled as promised and crawled all over the lawn on this one. Can’t wait to hear everyone’s responses!

      Like

    • skyyleracl

      AuditB,
      Great question. At the very least, auditors of the future have to understand not only IT, but security at a deeper level. You’re right about that. Auditors need to be technical and understand how to analyze data.

      I agree, a separate IT audit is nonsense, but with today’s auditors holding onto yesterday, it will be a while….

      Like

  3. Audit Monkey

    I will reply more fully in due course but I think there are several issues here. One, is the rate of change; it will vary from industry to industry. Second, you can have all the data analytics you like but you will need an Auditor to interpret the data and make sense of it. Depending on the circumstances and systems, I’m sceptical as to what it will show, e.g. a few immaterial misposted entries in the general ledger which no one cares about.

    Like

    • My dear monkey,
      It appears you’re only thinking in financial terms.

      Analytics applies to audits far beyond finances. I’ve created analytic scripts that search emails for specific terms and phrases to check the accuracy of a expensive, cloud-based system (the system flags salespeople who discuss prohibited topics on social media), catch salespeople who change credit accounts to their own street address and phone number, and identify addresses that are in reality PO boxes (not all PO boxes say “PO BOX”), identify data sold to us by vendors that have way too many people with birthdates that make them over 110 years old), and so on. It’s so much bigger than financial analysis.

      Here’s one for you. I also have a script that identifies split transactions. You enter the amount of the threshold and the time period to be reviewed and it identifies transactions that add up to or exceed the threshold, but were processed with X days (the time period you specified). I’d like to see you do that analysis on 1 million plus transactions without some automation.

      I think you need to broaden your view of analytics…

      Nevertheless, I look forward to your broadened reply.

      Like

      • Audit Monkey

        Of course I think in financial terms because I’m a qualified Accountant! I’ve been keen to do more computer based testing, or analytics since the mid 2000’s. However, using the examples above, my first question to the Business with the aged annuitants would be ‘what checks are there is the first instance to confirm claimants’ ages are valid?’. Doing a check of the data would be retrospective in nature and one could argue, too late in the day.

        I am slightly uncomfortable with your stance with analytics and it stems from the use of phrases such as “catching salespeople”. In my audits, I don’t set out to catch people out because there will always be clerical errors or minor deviations. Whether these are a cause for concern comes down to professional judgement or experience. In the main, I am of the view that the majority of people don’t go to work with the desire to defraud the firm but of course, it does occur.

        There is nothing new in the examples above; those transgressions (ignoring the advent of email), such as aged annuitants, dodgy salesmen, have been around before analytics, and will be around long after. As for split transactions, it does seem very public sector-ish and not my bag.

        Like

        • Audit monkey,
          Of course you focus on finances; but that was my point. Too often, finance auditors don’t look outside the box regarding what analytics can do. I would also venture that they often fail to look inside the finance box also. Analytics is so much more than just general ledger babysitting.

          I get the feeling that many auditors are threatened by analytics, and that it devalues their expertise, when in fact, it enhances it. Using your knowledge, you can cover more ground and new ground with analytics. Also, when non-IT auditors and IT auditors and analytic auditors team up, they are unstoppable. However, too many auditors (mostly finance and general auditors) brand analytics a fad and hope it goes away. Good luck with that.

          Almost all of the data analysis I do in the financial realm does NOT Involve general ledger entries, because fraud and errors often occur far before something hits the GL. Once it hits the GL, it is often obscured and has to be traced back. Why not use analytics to discover it at its source?

          Using analytics, we find many audit issues that we would never find through sampling in traditional audit methods, simply because we test every transaction, which may be a financial transaction or merely a log entry on a server or database.

          Yes, analytics and especially automation take more time than a simple sampling test. However, taking this extra time allows you to do tests you would never be able to do using traditional methods; and as mentioned before, you can test every transaction or event. Also, once you automate the test, you have more time to do other testing that you didn’t have before.

          Regarding incorrect claimant birthdates being retrospective, I agree. Never should have happened. But bad data occurs over and over. In this specific case we’re discussing, we bought the data. But data entry errors are made and even production data updates can also cause bad data. Again these should not happen. I don’t care where the bad data comes from, but bad data causes risk, and part of my job is to identify risk and it’s impact to the business. The fact that I’m finding the bad data retrospective doesn’t change the risk or the impact. It does make the business happy. Some Financial items we process depend on a customer’s birthdate.

          With regards to Fraud and “catching salesman”, The IIA requires auditors to consider the fraud angles in an audit and to test for just that. While the phrase I used definitely has undesirable connotations and smacks of audit being a policeman, the specific test I mentioned was requested by the business. I chose my words poorly but my analytic kicks UK arse. I do set out to identify people that may be committing fraud, making innocent errors, or making stupid errors.

          True, nothing new in the tests I described except for the fact I am testing millions of transactions instead of a sample. Except that I can quickly change the parameters of the test and run it again, meaning I can test several thresholds and whatnot. Except that I can automate those tests from the obtaining to the data, verifying of the data, cleaning of the data, transforming of the data, analyzing the data, and emailing me the results on a scheduled basis. All I have to do is open the email and research the findings.

          Split transactions are not your bag? So it’s OK if someone is spending more money than their authorized for, and they do that through several small transactions instead of one transaction that would get denied?

          Obtaining data and getting it into a usable format will always be a problem whether you’re doing sampling or analytics. That’s just another auditor excuse to not do hard work. If you can’t get data, that is an audit charter issue, as audit must have access to all data on demand.

          So I would argue the real problem in both cases is audit management. Audit department don’t request data early enough, don’t ensure the data is accurate early enough, and don’t ensure they have all the data they need early enough. It’s much easier to say we couldn’t get the data and it is to actually work to get the data.

          Auditors need to stop using the waterfall method to do audits; they need to start obtaining the data for several audits at once so that they get the data early enough to do all the verification of it before the audit testing clock starts ticking.

          In the end, as long as audit is focused on getting the audit done and not identifying risk, errors, inefficiency, and fraud, analytics and automation will suffer. Only when market forces require them to do otherwise will the bulk of auditors embrace the new audit manifesto.

          Eventually, when big failures occur and become public, the question won’t be where was internal audit, the question will be why didn’t internal audit analytics identify this?

          Either way, if audit can’t get the data, audit refuses to work differently to get the data, or are too lazy to do the hard work of obtaining and validating data, the chief audit executive (CAE) is to blame. Period.

          I still get the sense that you don’t see the value in analytics even though you really have not truly invested time exploring it.

          That’s why I lashed out at you and your alleged view of analytics.

          I’m still interested in hearing what tools, methods, and data record sizes you have analyzed.

          Like

    • skyyleracl

      Monkey,
      Mack came me a heads up on your comment, and frankly, he is being too kind. Which is why he seldom writes burning blog posts. And. I. Do.

      This is as nice as I can put it, for Mack’s sake: It’s auditors like you who hold advancement back, especially in technology, and specifically in data analysis. You need to graduate from the old school and join the modern age of audit and analytics.

      As Mack said, you not only need to expand your view, you need to gain more (some?) experience in data analysis, especially in regards to scripting and automating that analysis.

      In the past you have complained about auditors using Access database, and yet you yawn when data analysis is mentioned. I’d be interested to hear what great technology or analysis apps you’ve used in the past 2 years.

      Just because auditors as a whole haven’t embraced analytics, the business has, and auditors are being left behind. A majority of people used to believe the world was flat, and doctors used to work on cadavers and then deliver babies until they finally understood germs kills moms and babies.

      I tried to tone my comments down, and I was not very successful. In that way, you and I are more alike than you and Mack…

      Liked by 1 person

      • Audit Monkey

        I’ve complained about firms using Access as they’ve been burnt and it has been calamitous. Then to find an Audit Function using it, beggars belief. I do take issue with the comment that “It’s auditors like you who hold advancement back”. Don’t get me wrong, I’m all for analytics but often it comes back to that age old problem of getting hold of the data and it to be in a useable format. Based on experience, an awful lot of firms aren’t terribly good at this, so with the clock ticking and time drifting away, alternative audit procedures are adopted (as I would like to go home tonight!).

        The irony is I do see the role of the traditional Auditor diminishing. Firms aren’t happy to employ generalists anymore and want SME’s or properly qualified IT staff cum Auditors. It isn’t enough to be CISA qualified. Sure, there will be generalist jobs at the bottom of the pool but they won’t be paying much.

        Like

        • skyyleracl

          Monkey,
          I have never understood your adversion to Microsoft Access. If audit teams have been burnt by using it, that is the fault of the user, not the tool. If that’s the best tool you have, and you can use it effectively, I say use it. Access can still handle much more data than Excel can. Having said that, I would agree better tools exist.

          In his recent comment to you, Mack addressed using the old ‘can’t get the data’ excuse auditors have used effectively for decades. I agree with him totally. That all goes back to a CAE that is either weak or not providing adequate ‘audit supervision’.

          I understand that auditors have to get the audit done, and they do just that. But I’m calling on auditors to push back on their management to change how and WHEN data is requested, and to ask the CAE if the current state of datalessness is acceptable.

          I missed your irony. Do you mean that as IT and analytic auditors are more valued, the success of audit teams getting data hasn’t changed much? The value of traditional auditors is diminishing because they refuse to adapt to the new world. I don’t think much would change if every auditor were CISA qualified, as you, Mack, and I agree that the cert is a joke (but still required by the business, so auditors should still get it).

          One of 2 things will change the current situation: 1) the CAE stops accepting NO for an answer when data is requested, or 2) the analytic skills of the business surpass those of internal audit, and the audit team is made irrelevant.

          Lastly, Monkey, I appreciate your unwillingness to return tit for tat. We fired some flaming missiles at you, but you didn’t respond in kind. I am beginning to understand why Mack respects you so much. Well done.

          Like

  4. Pingback: Analytics Blog Debate Heating Up | ITauditSecurity

  5. Audit Monkey

    skyyleracl,

    Just to clarify when commented that “firms using Access have been burnt and it has been calamitous”, it wasn’t the Audit Functions but the Business. I’ve seen companies use Access which has been programmed by a ‘skilled amateur’, which has fallen over downline and shed loads of data have been lost. I appreciate that MS is working on a new version of Access which is supposedly more resilient. As for the data request, the big problem in the UK is finding the right people with the appropriate skills in IT to give you the data. It isn’t for the want of asking. To that end, IT are behind the curve, not necessarily the Audit Function.

    Like

  6. Pingback: IIA Analytics Article Dead Wrong | ITauditSecurity

  7. Pingback: 5 Things We Need from ACL in 2018 | ITauditSecurity

  8. Pingback: The Analytic Staircase for Auditors | ITauditSecurity

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.