Understand model and parameter risk
Discuss use of models in the overall ERM decision making process
Describe the development and use of models for decision making purpose in ERM
Discuss how the decision making process takes account of the org’s risk appetite
and corporate governance
and builds on the results of stochastic modeling
, scenario analysis
, stress testing
and analysis of model and parameter risk
Consideration of more general issues relating to modeling
Incl. model and parameter risk
How models can and should be used as part of the overall ERM process
Model is a mathematical representation of real-world processes
↪ Representation will be imperfect
↪ Incorrect of sub optimal decisions may be made
Model risk:
Risk arising from the use of an inappropriate or inaccurate model when assessing or managing risks
Parameter risk:
Use of inappropriate or inaccurate parameters or assumptions within such models
Stochastic uncertainty:
Arises from the randomness of a finite set of observations
As # of observation ↑, certainty in the model, parameters, and its prediction ↑
Stochastic models acknowledge this by producing a range of results
Due to limited observations, fitted parameters are never certain
Projections that assume only stochastic volatility around unchanging parameters will lead to the range of projection being too narrow
To allow for parameter uncertainty, use dynamic (simulated) parameters if the parameters can be modeled in some way
e.g. if least square regression has been used and a covariance matrix for the parameters is available
↪ Multivariate normal distn can be used to simulate the parameters, which themselves are used in stochastic simulations
Approach to determining the CI for the parameters by estimating a joint distribution for the parameters
fit a model using T data points
Simulate T data points using the model
Re-fit the model to the simulate data points
Record the parameter values
Repeat the process a large number of times, starting with the original data set each time
Obtaining a joint distribution for the parameters this way means that rather than using a static set of parameters to carry out the simulations, dynamic parameters can be used instead
Three assumptions that can be made in relation to the choice of model
True model or class of model is known
Model used is a simplification of a known, more complex reality
Model used is an approximation to an unknown, more complex reality
Most common in financial modeling
Caveat with the 3rd assumption as it can lead to the wrong model
Inappropriate projection of past trends due to
Errors in historical data
Incomplete data
Heterogeneity in the data, where the underlying drivers and their dependencies are not known or not projected separately
Selection of an inappropriate underlying distribution due to
Insufficient data
Not investigating a range of alternative candidate distributions
The number of parameters being chosen without reference to:
Need to avoid over simplification and the risk of implicit assumptions
Principle of parsimony:
Where there is a choice of fitted models, the optimal selection is the one with the fewest parameters as this should lead to more stable projections
Application of statistical models in practical ERM context
Prime objective in building model:
Enable the actuary or risk manager to give an organization appropriate advice so that it can manage its risks in a sound financial way
Use in day-to-day running of the company
Provide checks and controls to the business
Use of model points
A very large number of individual data points might need to be brought together into a manageable number of relatively homogeneous groups
Groupings need to be made in a way that each policy in a group is expected to produce similar results when the model is run (i.e. homogeneous?)
Then we can have a representative single policy in each group to be run through the model ⇒ the result to be found ⇒ Scale up this result in order to give the result of the total set policies in the group
Model point: The representative single point in a group
A number of such “model points” can then be used to represent the whole of the underlying business
For generic actuarial model
Must be valid and sufficiently rigorous for the purpose to which it will be put and adequately documented
Model points
chosen must be such as to reflect adequately the distribution of the business being modeled
Components of the model
must allow for all those features of the business being modeled that could significantly affect the advice being given
Input parameter
should be appropriate to the business being modeled
Take into account the special features of the company
and the economic
and business environment
in which it is operating
Workings of the model
should be easy to appreciate and communicate
Results
should be displayed clearly
Outputs
should be capable of independent verification for reasonableness
Should be readily communicable to those to whom advice will be given
Model should be capable of subsequent development and refinement
Model must NOT be:
Overly complex
So that either the results become difficult to interpret and communicate
Too long or expensive to run (unless this is required by the purpose of the model)
Avoid the impression that everything an be modeled
Additional items for ERM
Model should be amenable to an analysis of the impact of parameter uncertainty or incorrectly specified parameter values
Should exhibit behavior in simulations that is consistent with the past
However should not exclude plausible future scenarios that might be quite different from historical patterns
Shortcomings of the model should be clearly stated
Summary: models should…
Be appropriate for purpose (with limitations recognized)
Be robust
Having been selected from various possible candidate models with a variety of structures
Be as simple as possible whilst meeting their purpose
Complex models can be difficult to check, maintain, communicate and may imply spurious accuracy and unjustified confidence
Be developed over time
With regular reviews identifying when parameters and /or structures need to change
Sometimes be avoided
When more importance is attached to other activities
(e.g. identification and management of hard to quantify operational risks)
Risk management models should:
Reflect the dynamics of the organization…
both now and as expected in the future
Should allow for wider external factors
Implies that modeling scope should be defined to be comprehensive across all important and well-defined risks
Different models may exits for component risks, but these need to feed into an overall modeling scheme
Overall objective is to achieve a balanced (not unduly exaggerated or smooth) modeling outcome
Reasons to build models for ERM decision making:
Pricing of products or service
Assess economic value of the company
Estimate possible volatility of future profits and earnings
Determine capital adequacy requirements (regulatory
and internal
)
Project future capital or solvency
Assess the effect of risk management and mitigation techniques on both profits
and capital requirements
Assess the effect of other strategic decisions (e.g. changes in investment or new business strategy)
Evaluate projects
Steps of developing and applying a model:
Specify the purpose of the investigation
Collect data and group or modify if necessary
Choose the form of the model, identify parameters and variables
Estimate the required parameters
and any correlations
between them
Check the goodness of fit
(fit different model if poor fit)
Ensure that the model is able to project all required outputs
(e.g. cash flow, etc and incl. interactions between them)
Run the model with the selected estimated variables (or stochastic variables)
Output in appropriate format
(e.g. summarized for stochastic models)
Assess the sensitivity of results to different deterministic variable values
Monitor the results of the model and its application and make refinements as required
If cash flow projection over a long time horizon is required
More frequently the cash flows
are calculated ⇒ more reliable the output from the model
On the other hand: Less frequent ⇒ model can run faster
Using models in the corporate decision making process involves:
Inputs: information
, data
, assumption
, parameters
Model calculations: cash flows
, projections
, simulations
Outputs: deterministic
, ranges/sensitivities
, stochastic distribution of outcomes
(as appropriate)
Review and discussion
Additional Considerations
risk policy
, appetite
and the overall use of judgement
(not just outputs)Formalized judgement
Can have quantitative statements of risk appetite
and utility functions
built into the modeling rules
Should NOT be seen as substitute for the intuition of decision makers
Market consistent analysis approach
Some argues that corporate decision makers should worry only about the risk preferences and other investment opportunities of the company’s owner
(i.e. the market consistent analysis approach)
Assumes that:
Investors can diversified away company specific risk
Risk premium will compensate for systemic risk
However in practice both external
and internal perspectives
are important
Economic Value Added Model
A possible model output upon which decisions are made
Economic Value:
PV of all future s/h profits on economic basis
EVA expressed as % of cost of capital:
Difference between the increase in economic value and the WACC
Positive incremental EVA means go ahead
But in practice, should consider more than one risk metric that are distinct and independent but not overly complex
Consideration of different options might lead to further clarification of an organization’s risk appetite
e.g. Model might identify a number of alternative strategies each of which lies on the efficient frontier and are within the risk tolerances
Selection of the preferred option may require explicit expression of a corporate risk preference
Qualitative aspects
Just as if not more important than quantitative
aspects for ERM based corporate decision making in practice
Reliance on quantitative models alone (esp. on one model and one specific calibration of that model) can be dangerous
Esp. where the limitations of the data
, parameters
or model used are not fully understood or appreciated
Many companies that failed during the economic downturn has theoretically sound quantitative risk models
More on use of capital model in corporate decision making in Module 30