Model Risk

What is Model Risk?

Model risk is the potential loss an institution may incur as a consequence of decisions that are principally based on the output of internal models as a result of errors in the development, implementation, or use of models.

Understanding Model Risk

Model risk’s become prominent and of serious concern following the advent of the use of sophisticated quantitative models in dealing with applications in major disciplines. The risk arises mainly because of potential errors in the models and appropriate usage and implementation of the model. The errors and inaccuracies can cause considerable monetary losses, poor organizational decision-making, and damage to institutional reputation.

Model risk arises principally because of two reasons:

• The model might have fundamental inaccuracies that produce erroneous results for its intended use.
• The incorrect or inappropriate use of the model.

What is a Model?

Since model risk is caused by the use of models, it’s appropriate to also define a model. A model is a quantitative system or mathematical representation that processes input data to derive quantitative estimates of different variables.

A model contains a set of variable assumptions and data for inputs, processes, outputs, and scenarios. It applies mathematical, statistical, financial, and economic data and techniques in a model. A model contains three major components:

1. Inputs: Data and assumptions of the model

2. Process: Processes that transform inputs into quantitative estimates

3. Reporting: Expression of estimates into valuable information for management

Sources of Model Risk

The following are some of the sources of model risk:

1. Data

Data used in a model may be inaccurate, incomplete, or distorted. It is crucial in developing an effective model; hence, flawed data has the potential to compromise the whole model.

2. Model implementation

The incorrect and/or incomplete implementation of a model can lead to inaccurate or erroneous results that can have adverse effects on model results and the organizational decision-making process.

3. Methodology

Statistical methodologies have their own errors, such as sampling errors and standard errors, that occur in regression modeling.

4. Parameters and assumption

Unrealistic and incorrect assumptions may alter the intended parameters of a model, thereby inducing risk. When fitting model parameters, an error may result in the calibration of the model.

5. Misuse

Inappropriate use of a model may invalidate an otherwise effective model.

6. Interpretation

Misinterpreting the results of a model brings on significant risk as a misinformed course of action is likely to be followed.

7. Inventory

Incomplete and inaccurate model inventory leads to model risk.

Model Risk Management (MRM) Framework

A good model risk management (MRM) framework should be crafted based on industry best practices and conform to regulatory guidelines. An authority to benchmark the MRM framework is the Supervisory Guidance on Model Risk Management (SR 11-07) from the U.S. Federal Reserve.

The MRM framework should include the following processes in its life cycle:

1. Modeling Risk Standards

Minimum standards should be set on the development of a model, and these standards need to be followed and respected. The internal standards should be at the same level or higher compared to regulatory standards such as Supervisory Guidance on Model Risk Management (SR 11-07).

The standards should encompass standards for data quality, model changes, model use, expert judgment, model methodology, model validation, documentation, external model data, and model reporting, among others.

2. Model Risk Appetite

After the establishment of a risk policy, it is prudent that a statement of the Board model risk appetite is well articulated for effective model risk management. Risk appetite is the amount of risk that an organization is prepared and capable of assuming in order to meet its desired objectives.

The level of risk appetite for model risk will depend on the purpose in which the model is applied. Model risk appetite should be stated in terms of risk tolerance and various relevant metrics such as aggregate quantitative risk exposure, the number of high risk-rated models, etc.

3. Model Risk Identification

It is necessary to identify the specific risks that affect the organization. An inventory of existing models should be completed to identify key model changes. The model inventory should categorize features such as the following (among others):

• Model name
• Description of the purpose of the model
• How the model is used
• Frequency of its use
• Model assumptions or inputs

4. Model Risk Assessment and Measurement

A quantitative and qualitative risk assessment needs to be carried out to assess the model risk of each model. The two approaches will derive an enterprise-wide risk assessment framework.

The quantification of model risk uses various model risk measurement approaches, or they can use operational risk style model approaches.  There are three main techniques to quantify risk notably:

• Sensitivity analysis – Changes in model assumptions and parameters and monitoring of changing outcomes
• Backtesting – Testing a model by using historical data and comparing the output to past results
• Challenger model – Comparing the results of a model with results from another alternative model using the same data

A quantitative assessment will measure and collate each distinct quantifiable model risk assessment through the use of appropriate correlation factors.

Qualitative risk assessment involves consideration for the model fit for purpose. The result will indicate model robustness, which will have an impact on the model risk rating. A qualitative assessment considers the use of qualitative metrics to measure risk in a model – notably model compliance with standards, cumulative model errors, the degree of model risk assessment, and other qualitative factors.

5. Model Risk Mitigation

Possible risk mitigation strategies may include the following:

• Changes in the model’s development process
• Carrying out supplementary model validation considering changes in the nature and structure of existing risks and the emergence of new risks that the organization is exposed to.
• Employment of independent expert judgments on model result interpretation as a result of model uncertainties.
• Model adherence and applicability to new risk regulations
• Model efficiency and applicability enhancement measures to reduce risk such as additional capital can help mitigate risk

6. Model Risk Monitoring and Reporting

The model risk monitoring and reporting function seeks to identify the following issues:

• Monitoring if model risk policy and risk appetite are being adhered to as per policy. The process will recommend if management intervention is required if there is a divergence.
• A material model inventory should be carried out on each individual model to measure if it is being used as per the MRM policy framework.
• Results of model risk assessment and validation should be analyzed, and corrective action should be taken on any weaknesses identified.
• An overview of new trends in model risk management and any other relevant matters.

Conclusion

In summary, an MRM framework should encompass the following:

• Minimum model risk management standards as per regulatory guidelines
• A clear statement of the Board’s model risk appetite
• A risk identification process to reveal model risks that the organization is exposed to and which ones require comprehensive management
• Quantitative and qualitative model risk assessment
• A comprehensive array of model risk mitigation strategies
• A model risk monitoring and reporting framework

The overall risk management framework is as good as its implementation and the people who use it. Hence, an organization should cultivate a good risk culture within the organization.

Additional Resources

CFI offers the Commercial Banking & Credit Analyst (CBCA)® certification program for those looking to take their careers to the next level. To keep learning and advancing your career, the following resources will be helpful:

• Credit Risk
• Basel Accord
• Risk Shifting
• Financial Modeling Software
• See all financial modeling resources