Comparison of credit scoring algorithms: artificial intelligence-based algorithms vs. traditional methods
Published: 2023, Issue 1/2023
Abstract. The aim of this study is to compare the performance of logistic regression, decision tree, and random forest on a real-world P2P (peer-to-peer) lending database. The peer-to-peer (or person-to-person, P2P) lending market facilitates financial transactions between borrowers and lenders. Credit risk management is traditionally based on credit scoring, which is used to estimate the probability of default. An appropriate credit scoring technique is crucial for the long-term success of both financial institutions and P2P lending platforms. The comparison of credit scoring algorithms is conducted based on data from Lending Club. The database spans the first quarter of 2017 and contains 563,847 observations and 23 different variables. Based on the results, we conclude that the random forest is the superior credit scoring algorithm in the case of the Lending Club database.
Keywords: credit scoring, peer-to-peer lending, logistic regression, random forest, decision tree
Introduction
According to Teply and Polena (2020), peer-to-peer (P2P) lending, as a FinTech financial innovation, is a new form of online financial intermediation that connects borrowers with lenders. Borrowers and lenders interact through online P2P lending platforms. P2P lending platforms can provide loans with lower intermediation costs than traditional banks, as their operating costs are lower due to their online presence. This enables the offering of more competitive terms to both borrowers and lenders. On average, borrowers pay lower interest rates on P2P lending platforms than they do for loans provided by banks. Lenders with well-diversified loan portfolios achieve higher returns than those from traditional savings accounts. These facts are making P2P lending increasingly popular for both borrowers and lenders.
It is also important to highlight the limitations of P2P lending in order to obtain a comprehensive picture of the lending market. According to Zhao et al. (2021), one significant limitation relates to risk management, as the assessment of borrowers' creditworthiness on P2P lending platforms is often less thorough than in the case of traditional banks, which represents a higher risk for lenders. Unlike traditional banks, the credit risk must be borne by the investors rather than the P2P platform (Stern et al. 2017). The liquidity risk of P2P lending is also important; investors do not receive their invested capital back immediately and must wait until the end of the loan term. Interest rate risk and platform risk also limit P2P lending, as the business models and financial positions of P2P lending platforms may change, posing a risk to investors. In addition to these, potential risk factors arising from platform failure, fraud, or cybercrime represent risks to both investors and platforms (Milne–Parboteeah 2016). The first P2P lending platform, Zopa, was founded in the United Kingdom in 2005. Numerous P2P lending platforms have emerged following Zopa, including more than 1,000 in China. In the United States, Lending Club is the most significant P2P lending platform, which was listed on the New York Stock Exchange in 2014. The value of the P2P lending market was $84.89 billion in 2020, and according to forecasts, it will reach a value of $578.03 billion by 2028.
With the growth of individual loan applications, the assessment of credit risk has become increasingly important for financial professionals and researchers alike. According to Ágoston (2022a), traditional methods, which dominated this field for a long time, are facing increasing challenges due to the changing economic environment and the abundance of data. New technologies, such as artificial intelligence-based algorithms, offer a promising alternative alongside traditional models.
In this study, I review new approaches and subsequently compare them with traditional methods in terms of efficiency and applicability. The aim of the analysis is to provide insight into how new technologies can serve as effective tools for credit rating institutions in risk management and decision-making. Using metrics constructed based on the confusion matrix, I compare the performance of decision trees, random forests, and logistic regression. Based on the evaluation of the models' performance, we answer the question of whether new technologies are capable of improving the efficiency and effectiveness of credit rating institutions in the lending decision-making process.
The study consists of four parts. The first chapter contains the literature review, which is followed by the data and methodology chapter, in which the database used in the analysis, its cleaning process, and the methods employed in the analysis are presented. This is followed by the analysis of the results, and finally, the drawing of conclusions.
Literature Review
In contrast to traditional bank lending, P2P lending represents a newer approach, in which a direct relationship is established between borrowers and investors through online platforms. These platforms offer lower costs and faster processes for loan applicants, while simultaneously providing opportunities for those investors who are willing to participate directly in financing. It is important for decision-makers to understand the advantages and limitations of both options in order to make effective choices between financing opportunities. The lending decision is similar in both types of financing sources (Kgoroeadira et al. 2023).
According to Berger et al. (2021), credit scoring is a process through which financial institutions or lenders decide whether a given borrower (an individual or a company) is capable of repaying the loan according to the agreed terms. This process helps lenders assess the risk involved in granting the loan and assists in making decisions regarding the terms under which the credit is granted, or whether it should be granted at all. According to Pang et al. (2021), credit scores allow lenders to determine the probability that a borrower will repay the loan according to the agreed terms. It is important for both lenders and borrowers that these credit ratings are reliable and accurate.
Numerous studies examine credit scoring algorithms and compare their performance (e.g. Fraisse–Laporte 2022; Estran et al. 2022; Ahmed 2023; Wang et al. 2023; Song et al. 2023). The literature distinguishes between two major groups of credit scoring algorithms: traditional methods and artificial intelligence-based credit scoring algorithms. According to Jagtiani and Lemieux (2019), there are several differences between the methods. Artificial intelligence-based methods are generally capable of processing larger and more complex datasets.
These algorithms can effectively exploit correlations and patterns latent within the data. They often deliver better performance than traditional methods, as they are capable of detecting more complex patterns. This can result in more accurate and efficient outcomes during the credit scoring process. The results of traditional methods are more easily interpretable and explainable, as they operate based on mathematical models.
Traditional credit scoring algorithms began to proliferate in the 1970s. These algorithms are generally based on statistical models and computer programmes that analyse customers' financial data and other factors to determine their creditworthiness. According to Fraisse and Laporte (2022), traditional methods are typically statistical models or rule-based systems that make decisions based on pre-defined rules or parameters. These methods are generally unable to adapt to new data or changing environmental conditions. In the field of credit risk, the most frequently used statistical methods are logistic regression and Lasso regression.
According to Dumitrescu et al. (2022), logistic regression employs probability-based classification to predict whether a customer will be able to repay the loan or not. Logistic regression can be used in lending platforms where the objective is to improve the identification accuracy of defaults, as well as to determine whether a loan is in arrears.
Artificial intelligence-based credit scoring algorithms have become widespread over the past 10 years (Zhou et al. 2019). These algorithms perform more complex data analysis and are capable of processing larger volumes of data and more diverse data sources, such as social media activity or online shopping habits. Fraisse and Laporte (2022) studied the differences between artificial intelligence-based algorithms and traditional methods in the context of credit scoring. The primary difference between the methods lies in their flexibility and the depth of data analysis.
Dzik-Walczak and Heba (2021) compared the performance of logistic regression and decision trees. The database used in the analysis was sourced from Lending Club and pertains to the period between 2011 and 2013. They established that logistic regression achieved better results than the decision tree-based credit risk model. Similarly, Jagtiani and Lemieux (2019) compared the performance of logistic regression and decision trees, confirming the former result.
Wang et al. (2022) examined the credit risk characteristics of individuals using logistic regression, as well as a discrimination model based on XGBoost.
To achieve more accurate results in distinguishing between defaults, XGBoost was employed. In the case of logistic regression, the AUC value was the highest and it possessed the greatest accuracy. They concluded that the XGBoost-based credit risk assessment model possesses very good default discrimination capability and robustness.
Zhou et al. (2019) examined a classification method based on a random forest algorithm and subsequently compared its results with an XGBoost algorithm. If the lender applies a very low acceptance rate—that is, assuming very little risk—the efficiency of the random forest decreases. This means that the method will be less accurate or reliable at lower acceptance levels. Even when taking cost sensitivity into account, the method's ability to identify defaulting borrowers was not significantly improved. Ágoston (2022b) compared the SVM method with neural networks and random forests. AdaBoost and Bagging were used to balance, analyse, and sort the data. The SVM achieved a classification accuracy more than 5% higher on the analysed database.
It was established that the progress in default prediction is primarily attributable to artificial intelligence and machine learning.
Modern algorithms, such as neural networks, transformers, and natural language processing (NLP) models, are playing an increasingly significant role in the field of credit scoring, as they represent a substantial advancement over traditional statistical and machine learning methods. Neural networks are capable of recognising complex patterns, which is useful during credit scoring where the interactions between numerous variables must be taken into account. Deep learning techniques, such as recurrent neural networks, enable the processing of large volumes of data and the uncovering of hidden relationships between data points, which is particularly advantageous for identifying non-linear relationships.
According to Wang and Xiao (2022), transformers, which were originally applied in natural language processing, are increasingly being integrated into the field of credit scoring as well. These models efficiently handle sequential data and are capable of identifying long-term dependencies, which provides the opportunity to account for time-varying data in the credit scoring process, such as the evolution of a client's credit history.
NLP models are particularly effective when unstructured data, such as textual reviews or customer complaints, must be analysed. These models allow such types of information to be integrated into the credit scoring process, thereby increasing the predictive power of the models. Through the analysis of textual data using NLP techniques, an even more comprehensive picture of a borrower's risk profile can be obtained, according to Alonso Robisco and Carbo Martinez (2022).
In recent years, research has concluded that in many areas of credit risk assessment, deep learning outperforms traditional machine learning methods, and ensemble classifiers perform significantly better than individual classifiers (Shen et al. 2021). The research by Xia et al. (2017) is based on ensemble learning, which is a combination of multiple techniques whose performance is superior to that of the individual techniques alone. Furthermore, they also discuss how these models can be used effectively in credit scoring.
Additionally, hybrid models, which combine various algorithms, are becoming increasingly popular due to their accuracy.
Rudin (2019) analyses the importance of transparency and explainability in modern credit scoring models, particularly in the case of deep learning algorithms, which often employ black-box methods. She emphasises the significance of explainable artificial intelligence (XAI), which can increase the confidence placed in credit scoring models.
Data and Methodology
In this chapter, I present the database used in the analysis and the data cleaning process, which includes the removal of irrelevant variables, variable transformation, the handling of missing and outlier values, and undersampling. Following the presentation of the data, I will also address the methods used in the analysis.
According to Wang et al. (2021), Lending Club is a peer-to-peer lending company that connects borrowers with investors through an online platform. It provides loans to individuals in need of personal loans between \$1,000 and \$40,000. Borrowers receive the difference between the total loan amount and the processing cost. Investors purchase bonds covered by the personal loans and pay service fees to Lending Club. The company shares data on all loans issued through its platform during certain periods.
Figure 1 illustrates the number of new loans listed on the Lending Club platform over recent years. The data demonstrates the dynamic growth of P2P lending.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
The number of invoices.
Table or figure – see the PDF version of the issue for its exact content.
The current number of overdue invoices.
Table or figure – see the PDF version of the issue for its exact content.
Description
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
For every variable, I examined both outliers and missing values. None of the variables contained missing values; however, outliers were identified in several variables. The handling of outliers is a crucial step in data analysis, as they can significantly influence the results of the modelling. Outliers are data points that differ substantially from the rest of the data and often distort statistical analyses, such as the accuracy and reliability of models. I defined outliers as values that exceed the sum of the upper quartile (Q3) and 1.5 times the interquartile range (IQR). In the case of the income variable, I filtered out values exceeding 170,900 USD (2,913 observations). Regarding the debt-to-income ratio, I filtered out values exceeding 44.39% for the monthly instalment variable, and values exceeding 23 for the number of accounts variable. For the revolving credit utilised amount, the threshold was 122.05 USD, and for the upper limit of the lender's credit limits, it was 82,800 USD. For the number of credit accounts held by the borrower, I filtered out values exceeding 5 credit accounts. Table 2 presents the variables, the thresholds used for outliers, and the number of observations exceeding these thresholds.
I removed the outliers from the database. Table 2: Analysis of outliers Variable annual_inc (income) dti (debt-to-income ratio with monthly instalment) open_acc (number of accounts) revol_util (revolving credit utilisation ratio) total_rev_hi_lim (upper limit of credit limit) total_cu_tl (number of credit accounts) delinq_amnt (number of delinquent accounts) mo_sin_rcnt_rev_tl_op (days since revolving credit limit opening)
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
The descriptive statistics for the quantitative variables are presented in Table 3.
The loan status variable had six values in the original database (fully repaid, written off, in progress, in grace period, 16–30 days overdue, 31–120 days overdue); from the perspective of the analysis, two of these were relevant (fully repaid and non-performing loans), and I removed the other four values from the database. Thus, I created a default variable from the loan status variable. The default variable is a binary variable: the value 0 denotes performing loans, while the value 1 denotes non-performing loans, as also shown in Table 4.
Following data cleaning, the database contained data for 13,182 customers who repaid the loan, as well as data for 3,904 non-performing customers.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Methodology
Following the cleaning of the database, I compared the performance of logistic regression, decision trees, and random forest.
Logistic regression
According to Chang et al. (2022), the most frequently used statistical method for estimating the probability of non-performance is logistic regression. The dependent variable is binary, with a value of 0 for performing customers and 1 for non-performing customers. The form of the binary logistic regression is as follows:
P(y = 1|x1, ... ..., xk) = T (beta0 + beta1x1 + ... ... + betakxk + epsilon),
Table or figure – see the PDF version of the issue for its exact content.
where $y$ is a binary dependent variable, taking values of 0 or 1. The $x$ variables are the independent variables, which the function $T$ maps to real numbers between 0 and 1, such that the estimated value lies within the interval [0,1]. The $\varepsilon$ denotes the error term. The following equation demonstrates how the function $T$ maps these values:
Table or figure – see the PDF version of the issue for its exact content.
Subsequently, $\pi(x) = P(y = 1|x)$ was defined, which is as follows:
Table or figure – see the PDF version of the issue for its exact content.
The coefficients will be estimated by maximising the log-likelihood function. According to Brimacombe (2016), the log-likelihood is a measure of the goodness of fit to the input data and indicates how likely it is that the model predicts the observed data.
According to Wilkinson et al. (2022), odds ratios are useful for interpreting the results of logistic regression. The use of odds ratios facilitates the interpretation of the extent and direction in which various independent variables influence the probability of non-completion.
The odds ratio of the $\beta_j$ coefficient can be calculated as follows:
Table or figure – see the PDF version of the issue for its exact content.
Odds ratios demonstrate how a unit increase in the independent variable $x_j$ influences the probabilities of the dependent variable $y$ being equal to 1. If the odds ratio is greater than 1, an increase in the $x_j$ variable will increase the probability that $y=1$. If the odds ratio is less than 1, an increase in the $x_j$ variable will decrease the probability that $y=1$.
Decision tree
According to Chang et al. (2022), a decision tree is a predictive model that employs a tree structure to classify or perform regression on data. A decision tree begins with a root node, which contains all available data. Subsequently, the algorithm selects an attribute, which is used to partition the data into different branches. These partitions depend on the value of the attribute, and thus the tree grows as we move downwards from the root to the leaves. The decision tree continues to partition attributes on every new branch, proceeding recursively down the tree structure. When it reaches a stopping criterion—such as the maximum depth of the tree, a certain number of data partitions, or other predefined conditions—it creates leaves.
These leaves contain the results of the classification or regression.
According to Kim et al. (2019), the advantages of a decision tree include simplicity and interpretability, as its results are easy to interpret and explain.
However, it must be considered that it may be prone to overfitting, particularly on complex datasets. Furthermore, the application of optimal parameterisation and measures against overfitting is essential to ensure effective performance when solving a given problem.
Random forest
According to Teply and Polena (2020), a random forest is an ensemble learning method aimed at addressing the problem that decision trees can easily become irregular patterns if they have multiple branches. This phenomenon can lead to overfitting, as the variance will always be high, even if the bias is low. Random forests construct values that differ minimally from decision trees to reduce the impact of variance. This minimal difference is manifested in the sampling. Generally speaking, predictions may be more accurate with more decision trees, which signifies better model stability. This, however, also implies longer training times; therefore, when achieving model performance, the balance between training time and model stability must be taken into account.
Methods used for comparing model performance
Several methods can be applied to compare and evaluate the performance of credit scoring algorithms, which help determine how accurate and reliable each algorithm is in predicting non-performing and performing customers. The methods used by me are the classification matrix, the AUROC (AUC), and the ROC curve.
The confusion matrix contains four elements upon which the performance of the algorithm can be measured. These four elements are as follows: the true positive rate, which is the proportion of customers who are actually defaulters and were correctly identified by the model; the true negative rate, which is the proportion of customers who are actually non-defaulters and were correctly identified by the model; the false positive rate, which is the proportion of customers who are non-defaulters but were incorrectly classified by the model as defaulters; and the false negative rate, which is the proportion of customers who are defaulters but were incorrectly classified by the model as non-defaulters.
Based on the confusion matrix, the following indicators can be calculated: accuracy, which is the proportion of all correctly classified customers relative to the total number of customers; sensitivity, which indicates the percentage of observations belonging to the positive class that are correctly identified; and specificity, which indicates the percentage of observations belonging to the negative class that were correctly identified.
According to Shi et al. (2019), the AUROC (AUC), or the Area Under the Receiver Operating Characteristic (ROC) curve, is an important metric for evaluating the performance of classification models. The ROC curve illustrates the sensitivity of the algorithm against the false positive rate at various threshold values. The area under the curve (AUC or AUROC) demonstrates how well the algorithm can distinguish between defaulting and non-defaulting customers. This value ranges between 0.5 and 1; the lower the value, the worse the model performs, and the closer it is to 1, the better the model performs.
Results
I examined the correlations between variables in order to determine which factors influence the default variable.
I used the chi-square test for qualitative variables and the t-test for quantitative variables. The results of the tests can be found in the appendix. The table displays the name of the variables, the value of the t or $\chi^2$ statistic, as well as the significance level (p). In cases where the significance level is less than 0.05, a relationship can be demonstrated between the given variable and default.
Training and test database
I randomly split the database into a training database and a test database. Following the split, 70% of the original database became the training database, and 30% became the test database. The models are estimated on the training database, and their performance is evaluated on the test database. The training database contains 9228 good customers and 2733 bad customers, while the test database contains 3954 good customers and 1171 bad customers.
Logistic regression estimation
In the first step, I estimated a stepwise logistic regression to predict the probability of non-completion using the student database. Stepwise logistic regression makes decisions step-by-step regarding which variables should be added to or removed from the model (Estran et al. 2022). Its use allows for the automatic optimisation and simplification of the model without the need to make manual decisions regarding each individual variable. Table 5: Factors influencing the probability of non-completion based on logistic regression
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
The variables grade, the number of currently active revolving credit lines (num_actv_rev_tl), home ownership (home_ownership), loan term (term), the upper limit of the total revolving credit limit (total_rev_hi_lim), the loan amount (loan_amnt), annual income (annual_inc), the minimum FICO score (fico_range_low), the number of enquiries (inq_last_6mths), the debt-to-income ratio (dti), the verification status (verification_status), the number of tax liens (tax_liens), and the loan purpose (purpose) were found to be significant. The results of the logistic regression are presented in Table 5.
I tested the predictive capability of the logistic regression on the test database. Classification efficiency was examined based on the confusion matrix.
Table 6 presents the confusion matrix, which shows the actual values and the values predicted by the logistic regression. Table 6: Confusion matrix: logistic regression Predicted Actual 0 Fulfilment (0) Non-fulfilment (1)
Table or figure – see the PDF version of the issue for its exact content.
Real
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Based on the analysis, the accuracy metric is 0.4968, which means that the logistic regression correctly classified customers in 49.68% of cases. The sensitivity metric is 0.3832, indicating that 38.32% of customers in the performing class were correctly identified. The specificity metric is 0.8804, signifying that 88.04% of non-performing customers were correctly identified. The positive predictive value is 91.54%, while the negative predictive value is 29.71%.
The ROC curve graphically represents the sensitivity and specificity of the classification model across various threshold values. Figure 2 illustrates the ROC curve for the logistic regression. The AUC, or the area under the curve, is 0.6318; thus, the model can be classified as having moderately good predictive performance.
Subsequently, I also estimated the logistic regression using WoE-transformed2 data. Using stepwise logistic regression, I determined that the loan amount (loan_amnt_woe), the term (term_woe), the sub-grade (sub_grade_woe), the length of employment (emp_length_woe), the home ownership type (home_ownership_woe), the annual income (annual_inc_woe), the verification status of income (verification_status_woe), the residence (addr_state_woe), the debt-to-income ratio (dti_woe), the earliest reported credit line (earliest_cr_line_woe), the minimum FICO score value (fico_range_low_woe), the number of inquiries (inq_last_6mths_woe), the number of credit lines (total_acc_woe), the upper limit of total available credit (total_rev_hi_lim_woe), as well as the number of active revolving credit accounts currently used by the borrower (num_actv_rev_tl_woe) proved to be significant in predicting non-performance. The essence of WoE transformation is to group the independent variables within a given dataset based on the dependent variable, which facilitates analysis and modelling (Siddiqi 2012). WoE values indicate the extent to which observations belonging to a given group carry information regarding the probability of the dependent variable. The higher the WoE value, the more significant and important the information provided by the group for the purposes of prediction.
The results are presented in Table 7.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Variable
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Actual
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Based on the analysis, the logistic regression with WoE transformation correctly classified customers in 50.26% of cases. The model correctly identified 39.25% of performing customers, while 87.45% of non-performing customers were correctly identified. The positive predictive value was 91.35%, whereas the negative predictive value was 29.89%.
The AUC value is 0.6335; thus, the model distinguishes between performing and non-performing customers on the given dataset similarly to the logistic regression estimated on non-transformed data. The ROC curve is shown in Figure 3.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Decision tree
In the first step, I employed undersampling for the decision tree. According to Moscato et al. (2021), undersampling makes the data more balanced, which helps ensure that the model does not merely optimise for the majority class. This improves the accuracy of the model, resulting in a much more precise ratio of correct classifier predictions. I created an undersampled training database consisting of 2,951 good customers and 2,733 bad customers.
I also determined the X-relative error value, which indicates how well the tree performs on the given dataset according to the analysis measured during cross-validation. The smaller the value, the better the resulting decision tree can be generalised. Subsequently, I pruned the decision tree using the obtained value; the result showed that if a loan was classified as rating A or B, the probability of default was 0.32, or 32%. In the case of loans with a C rating, the homeownership variable will also be taken into account. The decision tree pruned with the CP value can be observed in Figure 4.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
The undersampled decision tree correctly classified customers in 65.25% of cases. Specifically, 62.25% of performing customers and 66.14% of customers in the non-performing category were correctly identified. Following the classification matrix, I plotted the ROC curve, which is shown in Figure 5. The AUC value is 0.642; therefore, the model can be categorised as having moderately good classification performance.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Real
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Based on the analysis, the decision tree correctly classified customers in 65.25% of cases. The sensitivity index is 0.6225, and the specificity index is 0.6614. The positive predictive value is 35.25%, while the negative predictive value is 85.54%. Figure 6 shows the ROC curve for the decision tree. The AUC, or the area under the curve, is 0.5866; therefore, the model can be classified as having weak predictive power.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Actual
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
The decision tree correctly classified customers in 69.31% of cases. Of the defaulting customers, 52.35% were correctly identified. The specificity index is 0.7433, indicating that 74.33% of non-defaulting customers were correctly identified. The positive predictive value is 37.65%, while the negative predictive value is 84.04%. The ROC curve is shown in Figure 8.
The AUC value is 0.6334, meaning the model can be classified as having moderately good predictive performance. Thus, in the case of the decision tree constructed by adjusting the prior probabilities, I obtained a better result than when incorporating a loss matrix, but a worse result than during undersampling.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
The random forest correctly classified 77.40% of the clients. Specifically, 6.66% of non-performing clients and 98.28% of performing clients were correctly identified.
The AUC value is 0.7009, indicating that the model can be categorised as having good discriminatory power. Figure 9 shows the ROC curve for the random forest.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
In order to facilitate a better comparison between the models, Figure 10 displays the ROC curve for all estimated models.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Conclusions
P2P lending is becoming increasingly popular for both borrowers and lenders. With the growth of individual loan applications, the assessment of credit risk will become increasingly important. Those using traditional methods face growing challenges due to the changing economic environment and the abundance of data. New technologies offer a good alternative to traditional models. This study examines whether new technologies are capable of improving the efficiency and effectiveness of credit rating institutions in the credit decision-making process. I examined the performance of logistic regression, decision tree, and random forest using a classification matrix, the ROC curve, and AUROC. Based on the results, the random forest performed the best.
One limitation of the research is the quality and size of the dataset used, which may influence the accuracy and generalisability of the algorithms. The results support the fact that the parameterisation and fine-tuning of different algorithms also have a significant impact on the outcomes, which can be utilised to optimise and automate credit rating processes, particularly for financial institutions for whom risk management and the accurate assessment of creditworthiness are vital. The application of artificial intelligence-based algorithms can increase predictive accuracy and efficiency, thereby improving the quality of financial decision-making.
I compared the results of the study with findings from other articles. The best AUC value in the analysis was 0.700 in the case of the random forest. In the literature, particularly with modern machine learning models, we often encounter higher AUC values, which can be 0.8 or higher in the case of a well-optimised model. This may suggest that the model's sensitivity and specificity are not optimal. The 77.40% accuracy measured in the random forest model approaches the results found in the literature, but higher values are achievable with appropriate data management and model fine-tuning. In the literature, models with accuracy above 80% are frequently found, especially in the case of balanced data. For logistic regression and the decision tree, sensitivity values are relatively low (around 38–66%), while specificity is relatively high. It occurs in the literature that for models built on balanced data, both values are higher, indicating that the models are better able to identify both positive and negative classes.
The research could be improved by increasing the size of the dataset. With a larger database, models are capable of better pattern recognition, which can improve model performance. Balancing the data, for example through oversampling, undersampling, or the generation of synthetic data, can improve the model's sensitivity. The application of more advanced models can also help increase accuracy; deep learning techniques, such as transformer-based models, often perform better in recognising complex patterns. Combining traditional models with advanced algorithms may also lead to an increase in the accuracy of the results.
References
Ágoston, N. 2022a. Artificial intelligence and machine learning methods for estimating corporate insolvency. Statisztikai Szemle 100(6), 584–609. https://doi. org/10.20311/stat2022.6.hu0584 Ágoston, N. 2022b. A systematic literature review of foreign bankruptcy prediction methods. Vezetéstudomány 53(1), 69–89. https://doi.org/10.14267/VEZTUD.2022.01.06 Ahmed, S. E. 2023. Determinants of Credit Ratings and Comparison of the Rating Prediction Performances of Machine Learning Algorithms. In E3S Web of Conferences 409, 05013. EDP Sciences. https://doi.org/10.1051/e3sconf/202340905013 Alonso Robisco, A.–Carbo Martinez, J. M. 2022. Measuring the model risk-adjusted performance of machine learning algorithms in credit default prediction. Financial Innovation 8(1), 70. https://doi.org/10.1186/s40854-022-00366-1
Berger, E. A.–Butler, A. W.–Hu, E.–Zekhnini, M. 2021. Financial integration and credit democratization: Linking banking deregulation to economic growth. Journal of Financial Intermediation 45, 100857. https://doi.org/10.1016/j.jfi.2020.100857 Brimacombe, M. 2016. Large sample convergence diagnostics for likelihood based inference: Logistic regression. Statistical Methodology 33, 114–130, https://doi. org/10.1016/j.stamet.2016.08.001 Chang, A. H.–Yang, L. K.–Tsaih, R. H.–Lin, S. K. 2022. Machine learning and artificial neural networks to construct P2P lending credit-scoring model: A case using Lending Club data. Quantitative Finance and Economics 6(2), 303–325. https://doi. org/10.3934/QFE.2022013 Dumitrescu, E.–Hué, S.–Hurlin, C.–Tokpavi, S. 2022. Machine learning for credit scoring: Improving logistic regression with non-linear decision-tree effects.
European Journal of Operational Research 297(3), 1178–1192. https://doi.org/10.1016/j. ejor.2021.06.053 Dzik-Walczak, A.–Heba, M. 2021. An implementation of ensemble methods, logistic regression, and neural network for default prediction in Peer-to-Peer lending.
Zbornik radova Ekonomskog fakulteta u Rijeci: časopis za ekonomsku teoriju i praksu 39(1), 163–197.
Estran, R.–Souchaud, A.–Abitbol, D. 2022. Using a genetic algorithm to optimize an expert credit rating model. Expert Systems with Applications 203, 117506. https://doi. org/10.1016/j.eswa.2022.117506 Fraisse, H.–Laporte, M. 2022. Return on investment on artificial intelligence: The case of bank capital requirement. Journal of Banking & Finance 138, 106401. https://doi. org/10.1016/j.jbankfin.2022.106401 Jagtiani, J.–Lemieux, C. 2019. The roles of alternative data and machine learning in fintech lending: evidence from the LendingClub consumer platform. Financial Management 48(4), 1009–1029. https://doi.org/10.1111/fima.12295 Kgoroeadira, R.–Burke, A.–Di Pietro, F.–van Stel, A. 2023. Determinants of firms’ default on unsecured loans in the P2P crowdfunding market. Journal of International Financial Markets, Institutions and Money 89, 101882. https://doi. org/10.1016/j.intfin.2023.101882 Milne, A.–Parboteeah, P. 2016. The Business Models and Economics of Peer-to-Peer Lending, ECRI Papers 11594, Centre for European Policy Studies.
Moscato, V.–Picariello, A.–Sperlí, G. 2021. A benchmark of machine learning approaches for credit score prediction. Expert Systems with Applications 165, 113986. https://doi.org/10.1016/j.eswa.2020.113986 Pang, S.–Hou, X.–Xia, L. (2021). Borrowers’ credit quality scoring model and applications, with default discriminant analysis based on the extreme learning machine. Technological Forecasting and Social Change 165, 120462. https://doi.org/10.1016/ j.techfore.2020.120462
Rudin, C. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence 1(5), 206– 215. https://doi.org/10.1038/s42256-019-0048-x Shen, F.–Zhao, X.–Kou, G.–Alsaadi, F. E. 2021. A new deep learning ensemble credit risk evaluation model with an improved synthetic minority oversampling technique.
Applied Soft Computing 98, 106852. https://doi.org/10.1016/j.asoc.2020.106852 Shi, B.–Zhao, X.–Wu, B.–Dong, Y. 2019. Credit rating and microfinance lending decisions based on loss given default (LGD). Finance Research Letters 30, 124–129. https://doi.org/10.1016/j.frl.2019.03.033 Siddiqi, N. 2012. Credit risk scorecards: developing and implementing intelligent credit scoring Vol. 3. John Wiley & Sons.
Song, Y.–Wang, Y.–Ye, X.–Zaretzki, R.–Liu, C. 2023. Loan default prediction using a credit rating-specific and multi-objective ensemble learning scheme. Information Sciences 629, 599–617. https://doi.org/10.1016/j.ins.2023.02.014 Stern, C.–Makinen, M.–Qian, Z. 2017. FinTechs in China–with a special focus on peer to peer lending. Journal of Chinese Economic and Foreign Trade Studies 10(3), 215– 228. https://doi.org/10.1108/JCEFTS-06-2017-0015 Teply, P.–Polena, M. 2020. Best classification algorithms in peer-to-peer lending. The North American Journal of Economics and Finance 51, 100904. https://doi.org/10.1016/j. najef.2019.01.001 Wang, D.–Chen, Z.–Florescu, I.–Wen, B. 2023. A sparsity algorithm for finding optimal counterfactual explanations: Application to corporate credit rating. Research in International Business and Finance 64, 101869. https://doi.org/10.1016/j.ribaf.2022.101869 Wang, C.–Xiao, Z. 2022. A deep learning approach for credit scoring using feature embedded Transformer. Applied Sciences 12(21), 10995. https://doi.org/10.3390/ app122110995 Wang, H.–Kou, G.–Peng, Y. 2021. Multi-class misclassification cost matrix for credit ratings in peer-to-peer lending. Journal of the Operational Research Society 72(4), 923934. https://doi.org/10.1080/01605682.2019.1705193 Wang, K.–Li, M.–Cheng, J.–Zhou, X.–Li, G. 2022. Research on personal credit risk evaluation based on XGBoost. Procedia computer science 199, 1128–1135. https://doi. org/10.1016/j.procs.2022.01.143 Wilkinson, J. D.–Mamas, M. A.–Kontopantelis, E. 2022. Logistic regression frequently outperformed propensity score methods, especially for large datasets: a simulation study. Journal of Clinical Epidemiology 152, 176–184. https://doi.org/10.1016/ j.jclinepi.2022.09.009 Xia, Y.–Liu, C.–Li, Y.–Liu, N. 2017. A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring. Expert Systems with Applications 78, 225–241. https://doi.org/10.1016/j.eswa.2017.02.017
Zhao, C.–Li, M.–Wang, J.–Ma, S. 2021. The mechanism of credit risk contagion among internet P2P lending platforms based on a SEIR model with time-lag. Research in International Business and Finance 57, 101407. https://doi.org/10.1016/j.ribaf.2021.101407 Zhou, J.–Li, W.–Wang, J.–Ding, S.–Xia, C. 2019. Default prediction in P2P lending from high-dimensional data based on machine learning. Physica A: Statistical Mechanics and its Applications 534, 122370. https://doi.org/10.1016/j.physa.2019.122370
Table or figure – see the PDF version of the issue for its exact content.
Table or figure – see the PDF version of the issue for its exact content.
Text extracted automatically from the PDF; for tables, figures and exact layout see the PDF of the issue.