This blog series explores how applying a set of coding best practices can help reduce errors during model development. We’ve captured these practices in the acronym MATHS: Modularized, Archived, Tested, Honed, and Standardized. This post focuses on the Archived element.
Consider this scenario:
You just spent the day writing code (modularized, of course!) for your new loss given default (LGD) model. A few of the scripts you worked on the previous day needed comments, so you added them in the morning. While doing that, you noticed a small bug in one of the scripts, so you fixed it. You then turned your attention to the bulk of your day’s work: finding relevant variables for the model. There was a lot of to-ing and fro-ing with that process, but you ended up finding multiple candidates. You were happy with the work. You made great progress. You turned off your laptop and left the office.
The next morning you arrive at your desk. In one hand is your morning’s first tankard of coffee; in the other, a donut the size of a small tire. You are excited about continuing the model work and hope your productivity is the same as the day before. As you go to place your coffee on the desk, the tankard clips the chair’s arm, and you bathe your laptop in your morning’s caffeine hit.
You panic because you know fluid plus electronics equals…then you hear it. PZZZT! The smell of ozone confirms what you heard. Your laptop is dead. And, with that, all your code from the previous day.
Do you:
- Announce loudly to the room “I’m out of here!” and then flip your desk over in frustration.
- Sob.
- Wish you had a software version control tool.
Archiving Code = Version Control
When we speak of the Archived best practice for model development, we are speaking about version control. This is a process that standardizes storage of code, helps keep track of changes in code, and offers code redundancy.
One way to visualize version control is to think of a stack of paper. The top sheet (1) is the current version of the code. The bottom sheet (3) is the first version of the code. All the sheets between these two are the different versions.
Suppose you wanted to go to the version of code that contained the log transformation for macroeconomic variables (2). In a proper version control tool, it is a simple matter of finding that version on the timeline and then retrieving it.
Benefits of Version Control
A benefit in having version control for model development is reducing the chance of unintentionally using out-of-date or buggy code. Once code is developed it should be tested to help find errors which can then be fixed before use. With version control, that updated code is archived and becomes the latest, go-to version. Without version control, it can be difficult to ensure you are reliably using the most up-to-date, bug-fixed version.
There are other benefits to having a version control in place. The first is version control helps enforce code readability. Have you ever written code that you didn’t need but you were afraid to delete it because it might be useful in the future? Instead of removing the code you just comment it out?
What this results in is a hodge-podge of code that is both relevant and irrelevant for the task. Not only is reading code like this confusing, but there is a risk of uncommenting the code that should have remained commented. Using version control allows you to save a version of the code with the potentially useful functionality and then save your next version without it.
The second is that version control can simplify collaboration. Version control products allow users to check-in and check-out code. As a result, multiple model developers can work on adding new, and updating old, modules used for model development without (usually) stepping on each other’s toes.
Reducing Model Risk with Archived Code
How does archived code reduce model risk? Using version control decreases model risk because it:
- Ensures tested code is readily available.
- Helps with code readability.
- Eliminates loss of code.
The first two effects reduce the chance the model developer uses conceptually or syntactically flawed code. The last effect eliminates the need to write new scripts, thereby removing the chance of writing conceptually flawed code in the future[1].
Common Approaches to Versioning Code
There are different ways to version your code. Some are better than others. Here are a few techniques we have seen or used ourselves:
- Manual suffixing
- We have all done it—applied a suffix to the end of a filename. As we work on a document, we will add something like “_v1”, “_v2”, etc. When it is final we might do a “_final”. Usually that works for a couple of days until another change comes in causing a need for a “_final_final” suffix.
- There are several issues with this type of versioning. The first is that collaboration quickly becomes cumbersome. Imagine yourself passing files to a coworker and the constant communication needed to ensure you are using the correct version. That could get confusing, and tiring, quickly. The second is that the files are on your computer so unless you are manually moving them to another location you run the risk of them evaporating if the computer dies.
- SharePoint version control
- If you have SharePoint—and enjoy using it—you can use the “Version History” functionality as a means of version control.
- This is better than manual suffixing because SharePoint is saving the files in a different location— you can retrieve the files even if you change computers. However, this is not ideal because identifying the previous version you want can be challenging and code collaboration would be frustrating.
- Git or other version control product
- We use Git at FRG. We love Git. And that’s all we are going to say about Git in this blog. There are many resources online that extol Git.
- There is a slight learning curve to use Git and the first time you need to merge two files tears might be involved. But there are many, many resources online to help. One of our favorites is this one.
What are some of the features a version control product offers?
Here is a sample of some of the features a version control product allows you to do:
- Create a repository (i.e., a centralized storage location for the code)
- Compare versions.
- Revert to a previous version.
- Branch code (i.e., create separate streams for development).
Conclusion: Why Version Control Matters
To recap, the Archived best practice is all about version control. Code versioning allows you to keep copies of your code, make changes to it as needed, and simplifies collaboration. Find the right process that works for you and your team. Embrace it. You will be glad you did the day you log onto your computer and face the “blue screen of death”.
Jonathan Leonardelli, FRM, Director of Business Analytics for FRG, leads the group responsible for business analytics, statistical modeling and machine learning development, documentation, and training. He has more than 20 years’ experience in the area of financial risk.
[1] In reading the Modularized blog and this blog you may see a pattern emerge with the model risk reduction. All the best practices captured in MATHS either relate to making sure the code does what the model developer wants it to do or decreasing the need to write new code, or both.
RELATED:
Model Risk Management: Reduce your Model Risk through Coding Best Practices