The housing market has undergone quite a change in the past decade, with more stringent lending criteria for housing having been enforced.

A key objective of financial institutions is to minimise the risk of mortgage lending by ensuring that the debtor is ultimately able to repay the loan.

In this example, multilevel modelling techniques are used to analyse data from the Federal Home Loan Bank System to determine the main influencing factors on loan-to-value ratios (LTVs) across the United States.

*As a disclaimer – the below is not intended as any form of financial advice and is merely used as an example to illustrate the workings of the relevant models.*

## Feature Selection – ExtraTreesClassifier

The dataset in question is **55990 rows × 82 columns**, and the first task is to conduct feature selection to ensure that the most relevant explanatory variables are selected for further analysis.

To this end, the ExtraTreesClassifier is run in Python – the script is available in the GitHub repository.

The ExtraTrees Classifier identified **Income (x9)**, **UPB or Unpaid Principal Balance (x12)**, **Loan Amount (x50)**, **Back-End Ratio (x52)**, and **PMI or percent of original loan balance covered by mortgage insurance (x55)** as the variables with the highest score as ranked by the classifier:

## Linear Regression

A linear regression was then generated in R to analyse the impact of the above variables in LTV.

Due to the VIF detecting multicollinearity across the variables for PMI and Back-End ratio, these two variables were dropped from the model.

Given that LTV is expressed in percentage format, the log of Income and UPB was used as the independent variables in the regression model.

The following results were generated:

`> # Linear Regression > reg1 <- lm(LTV ~ log(Income) + log(UPB), train)> summary(reg1) `

Call:

lm(formula = LTV ~ log(Income) + log(UPB), data = train)

Residuals:

Min 1Q Median 3Q Max

-0.78943 -0.06494 0.02240 0.08982 0.42595

Coefficients:

Estimate Std. Error t value Pr(>|t|)

(Intercept) 0.916374 0.015363 59.65 span class="hljs-number">2e-16 ***

log(Income) -0.061727 0.001654 -37.32 span class="hljs-number">2e-16 ***

log(UPB) 0.045761 0.001569 29.17 span class="hljs-number">2e-16 ***

---

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.1536 on 44789 degrees of freedom

Multiple R-squared: 0.03078, Adjusted R-squared: 0.03074

F-statistic: 711.3 on 2 and 44789 DF, p-value: span class="hljs-number">2.2e-16

From the above, the selected variables show high significance at the 5% level. However, the R-Squared of 15% is quite low. This indicates that while these variables are of importance in explaning the variation in LTV, there are either some important variables that are missing in this regard, or LTV itself is subject to a high degree of **random** variation.

Given that this is the case, the eventual multilevel modelling will not be used for **predicting** LTV per se – rather to analyse whether significant differences exist across this variable by state.

The Breusch-Pagan test for heteroscedasticity was run, and the VIF test for multicollinearity was run once again:

> library(lmtest)> bptest(reg1) studentized Breusch-Pagan test

data: reg1

BP = 1229.1, df = 2, p-value < 2.2e-16

> library(car)

> vif(reg1)

log(Income) log(UPB)

1.828859 1.828859

From the above, a VIF < 5 indicates that multicollinearity is not present in the model. However, a p-value of below 0.05 for the Breusch-Pagan test indicates that heteroscedasticity is present in the model. This is an issue across this dataset, given that different states with varying incomes are being represented. This results in a situation where states with higher incomes and higher housing prices are being given more weight by the model, skewing the results.

## Multilevel Modelling with lmer

In order to mitigate this issue, a better solution is to analyse LTV variation by State, specifically filtering by **FIPSStateCode** as illustrated in the dataset.

In this regard, random effects are firstly being tested for. It is hypothesised that the effect of Income, UPB and Loan Amount will vary across different states. Therefore, random slopes and intercepts are being generated in this regard.

`> # Random effects> r1 <- lmer(LTV ~ 1 + (1 + log(Income) |FIPSStateCode), data = train, REML=FALSE)Warning message: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :`

Model failed to converge with max|grad| = 0.0179241 (tol = 0.002, component 1)

> r2 <- lmer(LTV ~ 1 + (1 + log(UPB) |FIPSStateCode), data = train, REML=FALSE)

> r3 <- lmer(LTV ~ 1 + (1 + log(Amount) |FIPSStateCode), data = train, REML=FALSE)

Warning message:

In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :

Model failed to converge with max|grad| = 0.0318744 (tol = 0.002, component 1)

> anova(r1,r2,r3)

Data: train

Models:

r1: LTV ~ 1 + (1 + log(Income) | FIPSStateCode)

r2: LTV ~ 1 + (1 + log(UPB) | FIPSStateCode)

r3: LTV ~ 1 + (1 + log(Amount) | FIPSStateCode)

Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)

r1 5 -43118 -43075 21564 -43128

r2 5 -43906 -43862 21958 -43916 787.748 0 span class="hljs-number">2.2e-16 ***

r3 5 -43929 -43886 21970 -43939 23.235 0 < 2.2e-16 ***

---

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

For models **r1** and **r3**, a “Model failed to converge” error was obtained. Moreover, when testing for ANOVA, we see that the fit improvement between **r2** and **r3** is marginal. Therefore, **r2** is used to represent the random effect for the model.

Now, an attempt is made to model the fixed effects between Income and LTV.

`> # Fixed effects> m1=lmer(LTV ~ log(Income) + (1 + log(UPB) |FIPSStateCode), data = train, REML = FALSE)Warning message: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :`

Model failed to converge with max|grad| = 0.264243 (tol = 0.002, component 1)

> summary(m1)

Linear mixed model fit by maximum likelihood ['lmerMod']

Formula: LTV ~ log(Income) + (1 + log(UPB) | FIPSStateCode)

Data: train

AIC BIC logLik deviance df.resid

-45886.6 -45834.4 22949.3 -45898.6 44786

Scaled residuals:

Min 1Q Median 3Q Max

-5.5590 -0.4516 0.1039 0.6687 3.1559

Random effects:

Groups Name Variance Std.Dev. Corr

FIPSStateCode (Intercept) 1.395926 1.1815

log(UPB) 0.008612 0.0928 -1.00

Residual 0.020841 0.1444

Number of obs: 44792, groups: FIPSStateCode, 53

Fixed effects:

Estimate Std. Error t value

(Intercept) 1.642752 0.019925 82.45

log(Income) -0.071496 0.001586 -45.09

Correlation of Fixed Effects:

(Intr)

log(Income) -0.935

convergence code: 0

Model failed to converge with max|grad| = 0.264243 (tol = 0.002, component 1)

However, a “Model failed to converge” error is obtained once again. In this regard, the random effects portion of the model generated earlier will be used to model this problem.

Here are the summary coefficients for the model:

`> summary(r2)$coefficients Estimate Std. Error t value(Intercept) 0.8032787 0.007135987 112.5673`

Here is a breakdown of the intercept and slope across states:

`> coef(r2)$FIPSStateCode log(UPB) (Intercept) 1 -0.048725940 1.424839137`

2 0.071810557 -0.129437630

4 0.048194732 0.144809114

5 0.009605831 0.686029830

6 0.079868545 -0.373252028

8 0.066271417 -0.058015008

9 0.002805582 0.825132568

10 0.041174114 0.313942756

11 -0.021533357 1.059306143

12 0.048449155 0.193163918

13 0.029536299 0.427508846

15 0.033478378 0.274425498

16 0.083027128 -0.230813954

17 -0.037538462 1.267887084

18 0.062788077 0.002831210

19 0.012487451 0.632244118

20 0.042927859 0.273238843

21 0.084679686 -0.278537734

22 0.013490445 0.638005090

23 0.196958251 -1.714481216

24 -0.064853734 1.666322971

25 0.092872449 -0.434507155

26 0.063390995 -0.008880011

27 0.010592391 0.649027798

28 -0.021592652 1.134230484

29 0.046409632 0.220361661

30 -0.015760240 1.003635842

31 0.030503923 0.412809350

32 0.128775010 -0.842500079

33 0.042681993 0.239718731

34 0.079823518 -0.273370374

35 0.064508955 -0.037793158

36 -0.026011440 1.070115946

37 -0.017808095 1.028135790

38 -0.028361889 1.180258974

39 0.059861306 0.032634645

40 0.084900070 -0.235779484

41 0.134953722 -0.918560447

42 0.009586766 0.690380057

44 0.018198628 0.597527446

45 -0.084074305 1.892673690

46 0.024345909 0.479053971

47 0.090823001 -0.344979523

48 -0.003117153 0.856093955

49 0.084008316 -0.283965571

50 0.084902254 -0.165635531

51 -0.006251239 0.886143183

53 0.041150673 0.227720105

54 0.019188765 0.541915533

55 0.005422513 0.753196009

56 0.085633968 -0.296857819

66 0.118784559 -0.560868823

72 -0.055580831 1.595271936

attr(,"class")

[1] "coef.mer"

The data is then aggregated (i.e. the average UPB, LTV, Income, and Loan Amount is calculated), and the following data frame is generated along with the random slope and intercept across states:

When analysing the data frame as a whole, it is quite interesting to observe that there is generally an inverse relationship between the Intercept and Slope, i.e. the lower the intercept, the higher the Unpaid Principal Balance across states, and vice versa.

Here are some visualization plots:

**UPB vs LTV**

**LTV vs Income**

**UPB vs Income**

## Key Findings

From the plots, we can see that:

- There is a negative correlation between UPB and LTV, i.e. as the average LTV falls, the average UPB rises.
- There is a negative correlation between LTV and Income, i.e. as the average income rises, average LTV falls.
- There is a strong positive correlation between UPB and Income, i.e. as the average income rises, average UPB rises.

These findings are interesting in that they illustrate higher incomes are not necessarily the main predictor of risk when extending home loans. In the case of states with higher incomes, we have seen that the Unpaid Principal Balance rises quite significantly, reflecting a combination of 1) higher house prices in that state, and 2) the propensity of higher-income earners to purchase homes with significantly higher prices than that of lower-income earners.

Therefore, while states with higher incomes had slightly lower LTVs, the effect was not overly pronounced.

In this regard, one possible implication of this is that an institution can choose to set parameters as to their lending criteria, depending on how conservative the institution wishes to be.

For instance, an institution could choose to focus its marketing efforts on states with a UPB of less than **$300,000** and an LTV of lower than **0.75**.

Or, if the institution wanted to take some risk in terms of the loan size, another criteria set could be a UPB of up to **$600,000** but an LTV of lower than **0.70** to compensate for the added risk.

## Conclusion

In this example, we have seen how multilevel modelling is used to conduct regression analysis across different segments in a dataset, and specifically how this has been applied in assessing mortgage risk. The original post and GitHub repository can be found here.