FiinGroup Credit Score Methodology

The FiinGroup approach

We develop FiinGroup Credit Score by using a complex process that involves several key steps to ensure accuracy and reliability. These steps include defining requirements, developing the model, implementing the model in the production stage, and monitoring its accuracy over time.

Major steps involved in the model development process:

• Requirement definition and model design: This step ensures that the model supports its strategic objective by defining the outcome to be predicted, on what kind of population it is developed, which predictive data is used, and what modeling methodology is applied.

• Data engineering: we prepare appropriate data for the model development by defining a suitable sample; collecting raw data; splitting the sample into three parts for development, testing, and validation; ensuring high data quality by identifying and cleaning issues with the data; and aggregating granular data.

• Model assembly: This involves developing the actual algorithm. This step includes several sub-steps, namely the exclusion of inappropriate records based on logical criteria, the development of new features, the elimination of features that are useless or redundant, an initial estimate of the model coefficients, their iterative tuning, the calibration of the model outputs and decision-rules around them, and the documentation of the model.

• Model validation: is a governance process to independently ascertain the model’s fitness for use. Model implementation deploys the model in actual business operations; this involves feeding data inputs into the model and linking model outputs to business decisions.

• Continuous monitoring and improvement: We continuously monitor the accuracy of the model and make improvements as necessary to ensure that the FiinGroup CreditScore remains consistent in predicting power over time.

How is the FiinGroup CreditScore calculated?

The back-testing process: Models are designed to reflect reality. Back-testing compares real results with model-generated risk measures, we can use back-testing to develop a new model and to reassess the accuracy of existing models.

1. In the first stage, we perform feature selection, the process of using a single data field “predictors” to estimate the probability of the event. This is the most time-consuming process in model development.

This step consists of creating histograms for both categorical and numerical variables and then, given the default rates (bad/good) among the different groups within a variable, estimating the predictive power of that variable alone. To estimate the predictive power of the variable, analyze its correlation to the “default rate.”

1. Model Selection: Once the relevant variables have been identified, the next step is to select the most appropriate statistical model among a collection of candidate machine-learning models for a training dataset. Generally, several iterations, where different variable combinations are tested, will be needed until the appropriate model is found. The model that shows the higher predictive power out-of-sample should be selected, provided that all coefficients and variables make business sense.

2. The model evaluation process is a crucial step in ensuring the accuracy and effectiveness of the Credit model The objective of this step is to determine the model's generalization error, which measures the model's ability to make accurate predictions on unseen data.

This can be achieved through various evaluation metrics, such as a graphic distribution of the actual performance (good and bad) across the predicted probabilities of default. The greater the separation between the distributions, the more accurate the model is.

It is important to note that the model evaluation process does not end with the determination of the generalization error. Further fine-tuning and improvement may be necessary to ensure that the model continues to perform optimally in production.