5.4.5 - Interpreting Measures - Teradata Warehouse Miner

Teradata Warehouse Miner User Guide - Volume 3Analytic Functions

Teradata Warehouse Miner
Release Number
February 2018
English (United States)
Last Update

None of the measures described above is “best”; they all measure slightly different things. In the discussion below, product ownership association analysis is used as an example for purposes of illustration. First look at confidence, which measures the strength of an association: what percent of L customers also own R? Many people will sort associations by confidence and consider the highest confidence rules to be the best. However, there are several other factors to consider.

One factor to consider is that a rule may apply to very few customers, so is not very useful. This is what support measures, the generality of the rule, or how often it applies. Thus, a rule L R might have a confidence of 70%, but if that is just 7 out of 100 customers, it has very low support and is not very useful. Another shortcoming of confidence is that by itself it does not tell you whether owning L “changes” the likelihood of owning R, which is probably the more important piece of information. For example, if 20% of the customers own R, then a rule L R (20% of those with L also own R) may have high confidence but is really providing no information, because customers that own L have the same rate of ownership of R as the entire population does. What is probably really wanted is to find the products L for which the confidence of L R is significantly greater than 20%. This is what lift measures, the difference between the actual confidence and the expected confidence.

However, lift, like confidence, is much less meaningful when very small numbers are involved; that is, when the support is low. If the expected number is 2 and there are actually 8 customers with product R, then the lift is an impressive 400. But because of the small numbers involved, the association rule is likely of limited use, and might even have occurred by chance. This is where the Z score comes in. For a rule L R, confidence indicates the likelihood that R is owned given that L is owned. Lift indicates how much owning L increases or decreases the probability of the ownership of R, and Z score measures how trustworthy the observed difference between the actual and expected ownership is relative to what could be observed due to chance alone. For example, for a rule L R, if it is expected to have 10,000 customers with both L and R, and there are actually 11,000, the lift would be only 1.1, but the Z score would be very high, because such a large difference could not be due to chance. Thus, a large Z score and small lift means there definitely is an effect, but it is small. A large lift and small Z means there appears to be a large effect, but it might not be real.

A possible strategy then is given here as an illustration, but the exact strategy and threshold values will depend on the nature of each business problem addressed with association analysis. The full set of rules produced by an association analysis is often too large to examine in detail. First, prune out rules that have low Z scores. Try throwing out rules with a Z score of less than 2, if not 3, 4 or 5. However, there is little reason to focus in on rules with extremely high Z scores. Next, filter according to support and lift. Setting a limit on the Z score will not remove rules with low support or with low lift that involve common products. Where to set the support threshold depends on what products are of interest and performance considerations. Where to set the lift threshold is not really a technical question, but a question of preference as to how large a lift is useful from a business perspective. A lift of 1.5 for L R means that customers that own L are 50% more likely to own R than among the overall population. If a value of 1.5 does not yield interesting results, then set the threshold higher.