Model Risk Management and Validation

Introduction

Models are used throughout modern organisations to price products, assess credit, value assets, forecast demand, detect fraud and support strategic decisions. Increasingly these include machine learning and artificial intelligence models. When models are poorly designed, misused or not monitored, they can produce costly errors and regulatory breaches.

This Model Risk Management and Validation training course explains how to govern, validate and monitor models across their life cycle. Participants learn how to build a model inventory, tier models by risk, set development and documentation standards, perform independent validation, monitor performance and manage model changes and limitations.

The course draws on widely recognised supervisory guidance and industry practice, and is relevant to both financial institutions and other organisations that rely heavily on analytical models.

Course Objectives

By the end of this training course, participants will be able to:

• Explain the sources and consequences of model risk
• Design a model risk management framework and governance
• Build and maintain a model inventory with risk tiering
• Set standards for model development and documentation
• Plan and perform independent model validation
• Monitor model performance and manage limitations
• Address specific risks of machine learning and AI models
• Report model risk to senior management and the board

Target Audience

This training course is suitable for a wide range of professionals, including:

• Model risk managers and validators
• Quantitative analysts and model developers
• Data scientists
• Risk managers and chief risk office staff
• Internal auditors reviewing models
• Compliance and regulatory affairs professionals

Training Methodology

The course combines framework discussion with practical validation exercises. Participants tier a sample model inventory, review model documentation, design validation tests, interpret monitoring results and evaluate the governance of an AI model. Case studies illustrate model failures and lessons learned.

Target Competencies

• Model risk governance
• Model inventory and tiering
• Model validation techniques
• Performance monitoring
• AI model risk awareness
• Model risk reporting

Day One: Understanding Model Risk

• What counts as a model and why definitions matter
• Sources of model risk: design, data, implementation and use
• Consequences of model failure with real examples
• Supervisory expectations for model risk management
• The model life cycle from development to retirement
• Roles of owners, developers, users and validators
• Exercise: identifying models in your organisation

Day Two: Governance, Inventory and Standards

• Model risk management policy and governance structure
• Building a complete model inventory
• Risk tiering based on materiality and complexity
• Model development standards and documentation
• Approval processes and use limitations
• End-user computing tools and spreadsheets
• Workshop: tiering a sample model inventory

Day Three: Independent Model Validation

• Scope and frequency of validation by tier
• Conceptual soundness and assumption review
• Data quality and input testing
• Benchmarking, back-testing and sensitivity analysis
• Implementation and code review
• Validation findings, ratings and remediation
• Exercise: designing a validation plan

Day Four: Monitoring and Change Management

• Ongoing performance monitoring and thresholds
• Model drift, stability and population changes
• Overrides, overlays and management adjustments
• Model change control and revalidation triggers
• Managing model limitations and compensating controls
• Decommissioning and retiring models safely and traceably
• Exercise: interpreting a model monitoring report

Day Five: AI Models, Reporting and Assurance

• Specific risks of machine learning and AI models
• Explainability, bias and fairness testing
• Governance of third-party and vendor models
• Model risk reporting and aggregate risk profile
• Internal audit of the model risk framework
• Building validation capability and independence
• Personal action plan for model risk management