Model risk management begins with robust model development, implementation, and use [1].
Yep – this blog series from FRG is going to talk about model risk management (MRM). While those words may have you moving your mouse to the “close window” x, stay with us. We are only going to focus on a small section of the Model Development Lifecycle where our best practices will help you reduce model risk. Plus, we have a cool (albeit pretty nerdy) acronym for our best practices.
But before we get there, a little context to justify this series.
Model risk exists in the entire model development lifecycle. It is present at inception and only goes away when the bank decommissions the model[2].
When thinking about MRM many people are likely to think of the model validation process. Justifiably. Model validation is fundamental to MRM. However, proper development and implementation are also essential in reducing model risk. As the authors of SR Letter 11-7 note, that’s where it begins.
What is SR Letter 11-7?
SR Letter 11-7 is the US Federal Reserve Bank’s cornerstone guidance on managing model risk. It shaped how banks set up Model Risk Management (MRM) functions, requiring independent validation, documentation, and governance.
For this blog series, we are going to focus on ways to reduce model risk in model development. But not the entire model development process: just the code development aspect.
Best Practices for Code Development in Model Risk Management
While this is a very narrow area, we believe the best practices we explain can provide value to the model risk discussion because they:
- Decrease model risk due to helping build a robust model development process.
- Apply to any size bank or model development process, regardless of complexity.
- Reduce the need for new code creation, freeing modelers to do what they do best – use their business knowledge and experience to find the best model structure and variables to address the model objective.
- Result in faster model development.
FRG’s MATHS Framework for Model Risk Reduction
FRG’s best practices are captured in the acronym MATHS (oh yeah – we went there!)[3]:
- Modularized
- Archived
- Tested
- Honed
- Standardized
The remaining blogs in this series will provide explanations about these code development characteristics. Arguably, any code development should follow these best practices. Following them will lead to a more robust model development process and a reduction in model risk.
What is a model?
SR Letter 11-7 defines it as (for the purpose of the Letter): “…a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates”.
What is model risk in finance?
Model risk is the risk that arises when a model produces an inaccurate or incorrect result. For example, if a probability of default model predicts a low default probability on a new account, but that account defaults in the first quarter it is on the books, then that model is producing an inaccurate estimate.
Jonathan Leonardelli, FRM, Director of Business Analytics for FRG, leads the group responsible for business analytics, statistical modeling and machine learning development, documentation, and training. He has more than 20 years’ experience in the area of financial risk.
[2] It can enter at the beginning if the business objective isn’t defined accurately or communicated clearly.
[3] Is it math or maths? It’s both!