A couple of us were arguing about the differences between random, haphazard, and judgmental sampling. One person said that picking samples here and there manually was random sampling. I argued the method described was actually haphazard sampling. Another said that haphazard sampling was not appropriate and that “audit judgment” was valued, not haphazard sampling.
First, true random sampling requires the use of a pseudo-random generator, which can be a simple program, or in the case of ACL, you can use the Sample Records option.
Second, the difference in haphazard and judgmental sampling is huge. My colleague insisted that when you select a sample with no explicit method, that is judgmental sampling, and should be noted as such.
I disagreed and explained the difference this way: haphazard sampling was selecting samples from a population by merely picking one here and there without any criteria; judgmental sampling is selecting a sample based on some criteria.
For example, assume user access to SOX systems is being tested, and the population includes all users in the company. If users are selected haphazardly, the selection would most likely include users with no access to any systems (janitors), users with access only to email (mailroom clerks), and users with access to various SOX systems (IT, manufacturing, and marketing staff).
If users are selected judgmentally (the criteria being users most likely to have access to SOX systems), then the selections would be made from the IT, finance, management, and similar functions; janitors, mailroom clerks, and the like who have no such access would not be selected.
Speaking of haphazard selections, this article notes that haphazard sampling tends to be subconsciously biased, based on various studies conducted. People tend to select items that will reduce the workload (file folders in the top drawer) or are more attractive (such as items from the brightest colored bins). Originally I thought this bias would occur more often on the financial side than the IT side, but I can see how familiarity with the items sampled could impact sampling. In the case of selecting servers, one may subconsciously remember certain servers are located where controls are not as strong, administrators are more lazy, run more applications, or had multiple issues in years past. What do you think?
The article includes this guidance:
Auditors that continue to use haphazard selection should employ multiple debiasing procedures and carefully document these procedures in their workpapers. Such procedures might include a combination of: 1) stratification by time period, location, and dollar value; 2) use of a high-value top stratum where all items are audited; and 3) an increase in overall sample size. But auditors should understand that even these procedures will not correct for bias that results from bias-inducing factors that are not well controlled by stratification and practical increases in sample size (e.g., biases due to physical size, color, and number of adjacent neighbors). Ultimately, using random selection may be the more efficient way to avoid the cost and effort of debiasing procedures.
Read the article, How Reliable is Haphazard Sampling?
ITauditSecurity 