9.11 Model Testing

Important to test the model results against actual (see Meyers)

Challenges:

  • Don’t have underlying distribution to compare with

  • Can’t wait for 10 years to see how it forecast

General Insurance Reserving Oversight Committee test:

  • GIROC created triangles that met the assumptions of Mack and ODP

  • 30K 10 \(\times\) 10 triangles were created

  • Mack: losses were above the 99th percentile about 8-13% of the time \(\Rightarrow\) Mack underestimates tail events

  • ODP from England & Verrall: losses were above the 99th percentile about 3% of the time

    • This does not include the residual adjustments

Future testing:

  • Create datasets from claims transaction data and use them for model testing

9.11.1 Future Research

  • Test ODP bootstrap on realistic data from CAS loss simulation model

  • See how adjustments discussed here improve predictive power

  • Expand ODP bootstrap with

    • Munich Chainladder for incurred/paid

    • Claim counts and average severity

    • Different residuals (e.g. deviance or Anscombe residuals)

  • Select weights using Bayesian methods by AYs

  • Other risk analysis measures and use for ERM

  • SII requirements

  • Research in correlation matrix (difficult to estimate)