9.9 Diagnostics

Use diagnostics to judge the quality of the model:

  1. Test model assumptions

  2. Gauge quality of model fit

  3. Guide the adjustments of model parameters

5 diagnostics

  1. Residual graphs

  2. Normality test

  3. Outliers

  4. Parameter adjustment

  5. Model results

9.9.1 Residual Graphs

Plot residuals versus

  • CY, AY, Age, forecast loss (on x-axis)

  • Want to see random variability around zero

  • Bare in mind that we don’t have the same number of residuals at each point (helpful to plot the line for average as well)

Test assumption of iid residuals across the entire triangle

This helps with grouping for hetero adjustment

  • Plot the relative \(\sigma(r_{wd})\) for each group to further help with the groupings

  • Do all the plots again after adjustment

9.9.2 Normality Test

Normality is not required, only need this if we’re doing parametric bootstrap with normal distribution

Plot residuals against the normal best fit based on the percentiles

  • QQ-plot

Statistical tests:

  • Check if p-value > 5%

  • \(R^2\)

  • AIC

\[2p + n \left [ 1 + \ln(2\pi\dfrac{RSS}{n})\right]\]

  • BIC

\[n \ln\left( \dfrac{RSS}{n}\right) + p \ln(n)\]

  • \(RSS\) = actual residual - expected residual from normal then squared and summed

9.9.3 Outlier

Remove true outliers but do not remove points that are realistic extreme scenarios

Use box & whisker plot

  • Box hows 25%-tile to 75%-tile

  • Whiskers are 3 times the inter quartile range (both side total)

  • Residuals outside the range are graphed

9.9.4 Parameter Adjustments

Test model with different sets of parameters using GLM bootstrap

  • Check parameter significance based on t-statistics (>2)

Parameter selection process:

  1. Start with all the AY and Age parameters (\(\alpha_w\) and \(\beta_d\)) and remove the insignificant ones until only significant parameters are left

  2. Add CY parameter (\(\gamma\)) and check for significance

After selecting parameters:

  1. Check the diagnostics discussed above (e.g. residuals and normality)

  2. Make hetero adjustment if necessary

  3. Compare implied development with ODP

9.9.5 Review Model Results

Review outputs once we have decided on a model and run the bootstrap

  • Mean, s.e., CoV, Min/Max, and percentiles by AYs

  • Incremental fitted mean, s.e. and CoV for each cell in the triangle

    • Check for reasonability and consistency
Table 9.7: Model output review format
AY Mean Unpaid Standard Error CoV Min Max 50%-tile 75%-tile 95%-tile 99%-tile
(1) (2) (3) = (2) / (1) (4) (5) (6) (7) (8) (9)
1 - - - - - - - - -
\(\vdots\) - - - - - - - - -
\(w\) - - - - - - - - -
Total \(\sum\) - - - - - - - -

Remark. Standard Error: Col (2)

  • Total s.e. should be greater than any individual year but less than the straight sum of each AY’s s.e.

  • Expect s.e. to increase going down the column

  • This is different from Mack? Where we expect the total s.e. to be greater than the simple sum due to correlation between AYs? (questionable statement here)

Remark. Coefficient of Variation: Col (3)

  • Total CoV should be less than any individual year (due to diversification of results across AYs)

  • Except for the most recent AYs, CoV should decrease going down the column (due to larger based of unpaid losses for the more recent AYs)

  • Higher CoV for the most recent AYs due to:

    1. More parameters used to forcase for the most recent AYs \(\therefore\) parameter uncertainty \(\gg\) process variance

    2. Model maybe overestimating the uncertainty \(\Rightarrow\) Use BF of Cape Cod

Remark. Min & Max Col (4) - (5)