Introduction

The recent consultation paper¹ from the Bermuda Monetary Authority (BMA) proposed a range of enhancements to the regulatory and supervisory regime for commercial insurers and insurance groups, including new requirements in relation to modelling, governance, validation, stress testing and reporting. This consultation paper followed an earlier one from February 2023,² and is accompanied by field testing for (re)insurers.

Meeting these new requirements will be mandatory for (re)insurers that wish to use the scenario-based approach (SBA) for determining their best estimate liability (BEL).³ The rationale for this as noted in the consultation paper is that “The SBA is a tailored and dynamic approach and, therefore, quite demanding compared to other reserving approaches when it comes to implementation. In practice, this means that a significant investment is needed in governance, risk management systems, models and people.” The requirements, which are expected to come into force in early 2024, will apply to new applications to use the SBA as well as to existing grandfathered SBA models.

In this paper we focus on the enhanced model governance and validation requirements and the steps that (re)insurers need to take to be ready for the new regime.

BMA requirements

Governance and internal control requirements

Proposed requirements in these areas in relation to the SBA include:

1. Board approval of the initial use of the SBA and any major changes thereafter. Major changes need to be defined in advance and a second-line opinion is needed on whether the change requires board approval.
2. The board and management are responsible for ensuring the ongoing appropriateness of the design and operation of the SBA model, and for ensuring that it continues to be appropriate.
3. There needs to be a committee(s) in place to challenge new and ongoing model use, as well as model and assumption changes, and the use and reporting of model output. Model validation reports need to be discussed at committee level.
4. Policies are required, including at least a model risk management policy and a data quality policy. These policies should cover a number of aspects, including validation.
5. Roles of control functions need to be clearly defined in a number of areas, including validation.
6. Processes to identify and prevent conflicts of interest are required.
7. Ensuring that the systems, infrastructure and resources are adequate.
8. Ensuring that there are adequate and effective model controls in place.
9. Where outsourcing is used, the (re)insurer needs to demonstrate oversight and accountability as if the activities were performed internally. Outsourcing needs to be discussed with the BMA before implementation.

It is clear from the proposals that the board will need to have significant oversight of the SBA model and its governance, both at initial application and in ensuring its ongoing appropriateness. Policies may also need to be developed or enhanced and approved by the board. The board will also need to ensure that there are appropriate resources allocated and that controls are adequate.

Model risk management

The proposed requirement in this area is to develop a model risk management framework that includes the following aspects:

1. Model inventory
2. Model development, testing, implementation and use
3. Model limitations and uncertainty
4. Pre-model adjustments, in-model adjustments and post-model adjustments
5. Model validation
6. Model review, ongoing development and monitoring
7. Model risk reporting and deliberation
8. Use of and reliance on third-party models and outside experts
9. Model risk management audit

In our experience of model governance and risk management, a key starting point is having clarity on roles and responsibilities. Activities are typically split across the first and second line, with independence needed for activities such as validation. The Chief Risk Officer has an important role in establishing the model risk management framework and ensuring it meets the ongoing needs of the organisation.

Validation requirements

The BMA consultation paper defines model validation as “the set of processes and activities intended to verify that models perform as expected, in line with their design objectives, regulation and business uses…Effective validation helps ensure that models are sound.”

The proposed requirements in this area encompass:

• Model validation shall be performed by staff with appropriate incentives, competence, influence and authority so that there is an effective challenge mechanism.
• Insurers shall demonstrate that the model has been validated independently (externally or internally) by those who develop, change, update, run and use the model. Independence shall be demonstrated not just by the separation of lines but also by process, actions and outcomes.
• All SBA models shall be validated before use and at fixed intervals of at least three years thereafter or such other higher frequency considered appropriate by the insurer in line with its model risk management policy requirements.

Model validation can be performed either by external or internal resources. However, demonstrating independence is required to ensure there are no potential conflicts of interest.

The BMA consultation paper notes that the model validation process needs to at least specify the following areas:

1. Scope of validation:
1. Data and other input
2. Assumptions
3. Processing
4. Methodologies
5. Controls and governance
6. Model review process
7. Output and use
8. Documentation
2. Processes, methods and tools to be used
3. Frequency of validation
4. Model changes
5. Persons involved, roles, reporting lines and escalation paths
6. Output and reporting

“The model validation process shall apply to all SBA model components and cover all requirements. It shall also equally apply to models developed in-house and those purchased from or developed by vendors or consultants.”

The BMA consultation paper notes a range of processes that insurers could consider as part of the validation process. They can range from unit testing (e.g., sample model point) through to full independent model replications, as appropriate. In our experience, dynamic and static validations, along with sensitivity and stress testing, are important in gaining comfort over the robustness of the model.

The output from the validation should be a detailed model validation report that documents the validation process and the conclusion on the appropriateness of the model and the resulting technical provisions. The BMA proposals require that there is a clear reporting line to senior management and the board, a follow-up process for model validation findings, an action plan and implementation monitoring.

The BMA proposes that, where the validation identifies significant deficiencies, the model’s use might be constrained or potentially not allowed.

The model validation framework should include a definition of materiality for how the validator’s findings are classified (for example, a high-rated finding could be defined as one where the BEL is potentially misstated by more than x%). This will provide important guidance, although the experience of the validator is relevant given that there can be a significant element of judgement involved in carrying out the validation. One area where judgement can be required is when assessing the materiality of qualitative items and areas where a quantitative assessment has not been carried out.

As the third line of defence, internal audit has an important role to play in ensuring that there is appropriate challenge being provided by model validators, and that the model risk management framework is operating effectively. The BMA proposes that “internal audit should form its own independent opinion and provide assurance or otherwise on the adequacy of the model risk management activities performed by both the first-line and second-line functions.”

Conclusion

The new BMA requirements in relation to model governance and validation are substantial and (re)insurers need to assess them closely and plan for a smooth implementation over the coming months.

Whilst sound model governance and validation are generally already key aspects of a (re)insurer’s risk management framework, the new requirements may require significant additional effort to implement. This may create some additional overhead for (re)insurers. However, the benefit of regime strengthening—including a reduction in model risk and nasty surprises that can come from poorly designed or implemented models and model changes—will likely outweigh any additional cost.

How Milliman can help

Milliman has market-leading expertise in relation to model governance and validation and in understanding of the new BMA requirements for the scenario-based approach.

Our independent model validation services can be tailored to your requirements, such as: assurance of internal validation; benchmarking of model governance and validation frameworks against industry practice; or fully independent external validation.

Contact the authors or your usual Milliman contact for any queries.

Other related papers that might be of interest include:

• Structuring an Effective Model Risk Management and Validation Framework
• Risk and Financial Modelling
• Model Validation: Transforming Financial Models Into Powerful Decision-Making Tools
• Effective Model Validation Using Machine Learning
• IFRS 17 – Model Validation and Governance
• Validating the Black Box – Model Interpreters
• Proxy Model Validation
• Developing and Implementing Model Governance: Collaboration and Buy-In as Drivers of Successful Change

¹ The Bermuda Monetary Authority (BMA) has issued a consultation paper, : Proposed Enhancements to the Regulatory Regime for Commercial Insurers. See https://www.bma.bm/viewPDF/documents/2023-08-24-02-01-31-Consultation-Paper---Proposed-Enhancements-to-the-Regulatory-Regime-and-Fees-for-Commercial-Insurers.pdf. (bma.bm)

² Additionally, the BMA issued a consultation paper in February, “Proposed Enhancements to the Regulatory Regime for Commercial Insurers”. See https://www.bma.bm/viewPDF/documents/2023-02-24-12-48-11-Consultation-Paper---Proposed-Enhancements-to-the-Regulatory-Regime-and-Fees-for-Commercial-Insurers.pdf. (bma.bm)

³ Under the SBA, a (re)insurer uses an asset-liability management (ALM) model to assess the initial assets required to meet its liabilities under a base case and eight prescribed economic scenarios. The BEL is equal to the maximum of the initial assets required across these nine scenarios. A robust ALM model is needed to ensure the accuracy of the BEL.