Mahler.Credbility

Reading: Mahler, H. C., "An Example of Credibility and Shifting Risk Parameters"

Synopsis: Tested on the paper but not the appendices.

The paper deals with optimally combining different years of historical data when experience rating. The usual application of credibility, namely Data * Credibility + Prior Estimate * Complement of Credibility is used. The prior estimate of credibility is normally the average experience for a class of insureds but could also be the relative loss potential of the insured to the class average.

The goal is to understand how changes in the risk classification parameters over time results in older years of data adding less credibility to experience rating than expected.

Study Tips

To follow...

Estimated study time: x mins, or y hrs, or n₁-n₂ days, or 1 week,... (not including subsequent review time)

BattleTable

Based on past exams, the main things you need to know (in rough order of importance) are:

• fact A...
• fact B...

reference	part (a)	part (b)	part (c)	part (d)
E (2018.Spring #1)
E (2018.Spring #1)
E (2018.Spring #1)
E (2018.Spring #1)
E (2018.Spring #1)
E (2018.Spring #1)
E (2018.Spring #1)
E (2018.Spring #1)

In Plain English!

The efficiency of an experience rating plan is the reduction in expected squared error due to the use of the plan. The lower the expected squared error, the higher the efficiency.

Mahler uses data on baseball games to illustrate his points. He uses elementary statistics (binomial distribution, normal approximation) to conclude baseball teams do have significant differences between them over the years. Since they have differences between them, this means experience rating should predict future performance with some accuracy.

Next, Mahler asks if the differences in the win/loss record for a fixed team over time can be explained by random fluctuations from the same underlying distribution. He concludes the observed results cannot be explained this way, so the parameters of the distribution which describes the number of losses for a team in a year are changing over time.

One method for testing if parameters shift over time is the standard chi-squared test. Mahler groups the data into 5-year periods (other lengths could also be used) by team and uses the chi-squared test to measure if they could have the same underlying mean over the entire dataset. If a team didn't change its losing percentage over time, then its losing percentage should be normally distributed around its average. The chi-squared statistic is defined as [math]\displaystyle\sum_{i=1}^n\frac{\left(\mbox{Actual Loss}_i - \mbox{Expected Loss}_i\right)^2}{\mbox{Expected Loss}_i}[/math]. For a fixed team with n games, we have [math]n-1[/math] degrees of freedom if we have to estimate the expected loss.

Another method is to compute the average correlation between risks.

1. Fix the distance, t, between years.
2. Consider the set of all pairs of losing percentages which are t years apart for a fixed baseball team. Calculate the correlation coefficient for this dataset.
3. Repeat step 2 for each team and the average the resulting set of correlation coefficients for the fixed t value.
4. Repeat steps 2 and 3 for new t values.

For reference, the correlation coefficient is defined as [math]\frac{Cov(X,Y)}{\sigma_X\cdot\sigma_Y} =\frac{E[\left(x-\bar{x}\right)\left(y-\bar{y}\right)]}{\sigma_X\cdot\sigma_Y}[/math].

Since we are given a lot of years of baseball data, for small t values the set of pairs t years apart is relatively large. As t increases, the volume of data decreases. Mahler notes that a non-zero observed correlation is not necessarily statistically significant as the 95% confidence interval about 0 is approximately [math]\pm0.1[/math] and this increases as the number of data points decreases.

Looking at the results shown in the paper, we observe the correlation method yields correlations which are significantly greater for years that are closer together (small t values) than those further apart. In fact, after approximately 10 years there is no distinguishable correlation between years. This leads Mahler to conclude recent years can be used to predict the future.

Statement of the Problem

Let X be the quantity we want to estimate. Let Y_i be various known estimates (the estimators) and set X to a weighted linear combination of the estimators, i.e. [math]X=\sum_i Z_i\cdot Y_i[/math]. The goal is to find the optimal set of weights {Z_i} that produces the best estimate of the future losing percentage.

Four Simple Solutions

Mahler eventually covers six solutions to the problem. We'll follow in his footsteps and begin with four easy options.

1. Assume every risk is average, so set the predicted mean equal to 50%. This assumes all games have a winner/ties are negligible. It also ignores all historical data, so gives 0% credibility to the data.
2. Assume the previous year repeats itself. This gives 100% credibility to the previous year.
3. Credibility weight the previous year with the grand mean.
   • The grand mean is some external estimate that is independent of the data. In this case it is 50% because we assume equal likelihood of a win or loss.
   • Cases 1 and 2 are special cases of this, corresponding to 0% and 100% credibility respectively.
4. Give equal weight to the most recent N years of data.
   • This can be further extended by calculating the credibility, Z of the N years of data and then giving each prior year weight [math]\frac{Z}{N}[/math] and weight [math]1-z[/math] to the grand mean.

There are various choices for determining the credibility used in the fourth option. Bühlmann, Bayesian or classical limited fluctuation credibility methods could be used.

Three Criteria for Deciding Between Solutions

1. Bühlmann and Bayesian credibility methods which minimize the mean squared error.
   • The sum of squared errors (SSE) is defined as [math]SSE=\sum_{\mbox{team}}\left(\sum_{\mbox{years}}\left(X_{est,team}-X_{actual,team}\right)^2\right)[/math]
   • The mean squared error is defined as [math]\frac{SSE}{\# \mbox{ teams}\cdot\#\mbox{ years}}[/math].
2. Small chance of large error (classical credibility)
   • When the test is met, there is a probability P of a maximum departure from the expected mean of no more than k percent.
   • We can't directly observe the loss potential as it varies over time. Thus, averaging over time produces an incorrect result.
   • Rephrased - we're looking for credibilities which minimize [math]\mbox{Pr}\left(\frac{|X_{est,team}-X_{actual,team}|}{X_{est,team}}\gt k\%\right)[/math], where k% is some predetermined threshold.
3. Meyers/Dorweiler Credibility:
   • Calculate the correlation using the Kendall Tau statistic. The first quantity is the ratio of actual losing percentage to predicted losing percentage. The second quantity is the ratio of the predicted losing percentage to the overall average losing percentage.

Section 8 of Mahler's paper applies each of these methods in turn to the baseball data. It's best given a quick skim-read. One key takeaway is found in Section 8.3: As the number of previous years, N, used as estimators increases, the credibility of the entire N year period decreases when using Bühlmann or Bayesian techniques. This is counterintuitive at first because actuaries typically expect credibility to increase as the volume of data is increased. However, Mahler points out the result is actually what we should expect given the parameters are shifting significantly over time. This effect is also seen using classical credibility but isn't seen when Meyers/Dorweiler credibility is used.

A question considered by Mahler in Section 8.5 is what constitutes a significant reduction in the mean squared error when using Bühlmann or Bayesian techniques? In the appendices (Alice:"Remember you're not tested on the appendices..."), Mahler derives a theoretical limit for the best reduction in mean squared error that can be achieved by credibility weighting two estimates. The theoretical limit derived relates to the minimum of the squared errors resulting from placing either 100% weight or 0% weight on the data. When N years of data is used under the assumption that the distributional parameters do not shift over time, the best possible reduction in squared error is [math]\frac{1}{2\cdot(N+1)}\%[/math], so for [math]N=1[/math], a 25% reduction in squared error is possible, or as stated in the paper, the credibility weighted estimate has a mean square error at least 75% of that of the mean square error resulting from giving either 100% or 0% weight to the data. Mahler uses this to conclude his measured reduction of the squared error to 83% for the baseball data set is significant.

Since there is less benefit to including older data (due to shifts in the distributional parameters), delays in receiving the data result in a reduction in the accuracy of experience rating. As expected, the mean squared error increases rapidly at first and then tapers off (increases at a decreasing rate) as the data delay widens. Consequently, the credibility of the data decreases at a decreasing rate as the time delay between the latest available data and the prediction time increases.

More General Solutions

We now cover the last two method for assigning credibility weights.

Method 5: Single exponential smoothing

Here we assign geometric decreasing weights. The latest year gets weight Z, the next year weight [math]Z\cdot(1-Z)[/math], and continue in this fashion until the second to oldest year gets weight [math]Z\cdot(1-Z)\ldots\cdot(1-Z)^{n-2}[/math] while the oldest year has weight [math](1-Z)^{n-1}[/math]. Note that the weights sum to 1.

Method 5 is an example of a mathematical filter. The approach of giving a set of data credibility Z and the complement weight 1-Z is an application of this where [math]n=1[/math].

Method 6:

Generalizing method 5, we assign weights [math]Z_1,\ldots,Z_i[/math] to the first i years of data and then the balance, [math]1-\sum Z_i[/math] to the prior estimate, M.

Mahler recommends choosing M to be the grand mean, that is the mean of the entire dataset.

The more years of data used, the harder it is to determine an optimal set of weights [math]Z_1,\ldots,Z_n[/math] for this method.