Mahler 5
Posting a couple of questions received by email on this topic.
Questions:
- Why do you use claim counts instead of claim frequency?
- Why should we use the correlation test instead of the chi-squared test?
- The correlations initially decreased but started increasing at lags 7 and 8, likely due to thin data. However, the solution only mentions the decreasing trend. Is there a good cutoff point for the correlation lag? If not, should we mention the "increasing trend" on the exam?
Answers:
- We are using frequency but it's important to convert to claims when chi-squared testing. This is because I could have an expected frequency of 10% and an actual frequency of 9% and the impact is very different depending on the exposure associated. If there are only 100 exposures then the absence of a single claim is enough to get the actual frequency. Whereas if there are 100,000 exposures then an actual frequency of 9% means I'm "missing" 1,000 claims which is a much bigger deal as with more volume we should be able to estimate more accurately.
- You will get similar results if you perform the correlation test using claim counts or claim frequency. The claim frequency approach gives all years equal weight whereas the claim count approach is weighting by the number of exposures. The Mahler paper uses losing percentages i.e. frequency but assumes all teams play the same number of games so basically ignores this question about exposure weights. Given the potential for credibility concerns it seems like claim counts may be more appropriate to use. Using claim counts rather than claim frequency is also consistent with the chi-squared test which, if using claim frequency, must have the test statistic calculated as the exposure weighted average of the individual years.
- The chi-squared test can be passed by a single abnormal year whereas using the correlation test forces us to have the pattern consistently shifting over time. In other words, an outlier might cause the chi-squared test to be significant but the correlation test should be much less sensitive to this.
- You're correct about thin data likely causing the increasing trend for ages 7 and 8. I can't locate the reference at the moment but historically it has been recommended to use at least 3 or 4 data points before trusting a correlation figure. I think this is actually mentioned in a paper on the Exam 7 syllabus. So lag 7 is right on the cut-off and 8 contains insufficient data. The idea Mahler is trying to convey here is how much value we get from adding additional years of data. What we're seeing is the correlations decrease so we're not getting as much benefit from adding older data because the risk parameters are shifting. It's therefore highly unlikely to be useful to suddenly add in additional data that is much older/goes against the earlier trend observed. I think it would be reasonable to say the increase in correlations observed at lags 7 and 8 is likely irrelevant due to the small number of data points. However, I do not think it's necessary to say this on an exam to guarantee full credit.