+ - 0:00:00
Notes for current slide
Notes for next slide

Multiple Logistic Regression

1 / 23
2 / 23

Types of statistical models

response predictor(s) model
quantitative one quantitative simple linear regression
quantitative two or more (of either kind) multiple linear regression
binary one (of either kind) simple logistic regression
binary two or more (of either kind) multiple logistic regression
3 / 23

Types of statistical models

response predictor(s) model
quantitative one quantitative simple linear regression
quantitative two or more (of either kind) multiple linear regression
binary one (of either kind) simple logistic regression
binary two or more (of either kind) multiple logistic regression
4 / 23

Types of statistical models

variables predictor ordinary regression logistic regression
one: \(x\) \(\beta_0 + \beta_1 x\) Response \(y\) \(\textrm{logit}(\pi)=\log\left(\frac{\pi}{1-\pi}\right)\)
several: \(x_1,x_2,\dots,x_k\) \(\beta_0 + \beta_1x_1 + \dots+\beta_kx_k\) Response \(y\) \(\textrm{logit}(\pi)=\log\left(\frac{\pi}{1-\pi}\right)\)
5 / 23

Multiple logistic regression

  • ✌️ forms
Form Model
Logit form \(\log\left(\frac{\pi}{1-\pi}\right) = \beta_0 + \beta_1x_1 + \beta_2x_2 + \dots \beta_kx_k\)
Probability form \(\Large\pi = \frac{e^{\beta_0 + \beta_1x_1 + \beta_2x_2 + \dots \beta_kx_k}}{1+e^{\beta_0 + \beta_1x_1 + \beta_2x_2 + \dots \beta_kx_k}}\)
6 / 23

Steps for modeling

7 / 23

Fit

data(MedGPA)
glm(Acceptance ~ MCAT + GPA, data = MedGPA, family = "binomial") %>%
tidy(conf.int = TRUE)
## # A tibble: 3 x 7
## term estimate std.error statistic p.value conf.low conf.high
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 (Intercept) -22.4 6.45 -3.47 0.000527 -36.9 -11.2
## 2 MCAT 0.165 0.103 1.59 0.111 -0.0260 0.383
## 3 GPA 4.68 1.64 2.85 0.00439 1.74 8.27
8 / 23

Fit

What does this do?

glm(Acceptance ~ MCAT + GPA, data = MedGPA, family = "binomial") %>%
tidy(conf.int = TRUE)
## # A tibble: 3 x 7
## term estimate std.error statistic p.value conf.low conf.high
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 (Intercept) -22.4 6.45 -3.47 0.000527 -36.9 -11.2
## 2 MCAT 0.165 0.103 1.59 0.111 -0.0260 0.383
## 3 GPA 4.68 1.64 2.85 0.00439 1.74 8.27
9 / 23

Fit

What does this do?

glm(Acceptance ~ MCAT + GPA, data = MedGPA, family = "binomial") %>%
tidy(conf.int = TRUE)
## # A tibble: 3 x 7
## term estimate std.error statistic p.value conf.low conf.high
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 (Intercept) -22.4 6.45 -3.47 0.000527 -36.9 -11.2
## 2 MCAT 0.165 0.103 1.59 0.111 -0.0260 0.383
## 3 GPA 4.68 1.64 2.85 0.00439 1.74 8.27
10 / 23

Fit

What does this do?

glm(Acceptance ~ MCAT + GPA, data = MedGPA, family = "binomial") %>%
tidy(conf.int = TRUE) %>%
kable()
term estimate std.error statistic p.value conf.low conf.high
(Intercept) -22.373 6.454 -3.47 0.001 -36.894 -11.235
MCAT 0.165 0.103 1.59 0.111 -0.026 0.383
GPA 4.676 1.642 2.85 0.004 1.739 8.272
11 / 23

Assess

What are the assumptions of multiple logistic regression?

12 / 23

Assess

What are the assumptions of multiple logistic regression?

  • Linearity
  • Independence
  • Randomness
12 / 23

Assess

How do you determine whether the conditions are met?

  • Linearity
  • Independence
  • Randomness
13 / 23

Assess

How do you determine whether the conditions are met?

  • Linearity: empirical logit plots
  • Independence: look at the data generation process
  • Randomness: look at the data generation process (does the spinner model make sense?)
14 / 23

Assess

If I have two nested models, how do you think I can determine if the full model is significantly better than the reduced?

15 / 23

Assess

If I have two nested models, how do you think I can determine if the full model is significantly better than the reduced?

  • We can compare values of \(-2 \log(\mathcal{L})\) (deviance) between the two models
15 / 23

Assess

If I have two nested models, how do you think I can determine if the full model is significantly better than the reduced?

  • We can compare values of \(-2 \log(\mathcal{L})\) (deviance) between the two models
  • Calculate the "drop in deviance" the difference between \((-2 \log(\mathcal{L}_{reduced})) - ( -2 \log(\mathcal{L}_{full}))\)
15 / 23

Assess

If I have two nested models, how do you think I can determine if the full model is significantly better than the reduced?

  • We can compare values of \(-2 \log(\mathcal{L})\) (deviance) between the two models
  • Calculate the "drop in deviance" the difference between \((-2 \log(\mathcal{L}_{reduced})) - ( -2 \log(\mathcal{L}_{full}))\)
  • This is a "likelihood ratio test"
15 / 23

Assess

If I have two nested models, how do you think I can determine if the full model is significantly better than the reduced?

  • We can compare values of \(-2 \log(\mathcal{L})\) (deviance) between the two models
  • Calculate the "drop in deviance" the difference between \((-2 \log(\mathcal{L}_{reduced})) - ( -2 \log(\mathcal{L}_{full}))\)
  • This is a "likelihood ratio test"
  • This is \(\chi^2\) distributed with \(p\) degrees of freedom
15 / 23

Assess

If I have two nested models, how do you think I can determine if the full model is significantly better than the reduced?

  • We can compare values of \(-2 \log(\mathcal{L})\) (deviance) between the two models
  • Calculate the "drop in deviance" the difference between \((-2 \log(\mathcal{L}_{reduced})) - ( -2 \log(\mathcal{L}_{full}))\)
  • This is a "likelihood ratio test"
  • This is \(\chi^2\) distributed with \(p\) degrees of freedom
  • \(p\) is the difference in number of predictors between the full and reduced models
15 / 23

Assess

  • We want to compare a model with GPA and MCAT to one with only GPA
glm(Acceptance ~ GPA, data = MedGPA, family = binomial) %>%
glance()
## # A tibble: 1 x 7
## null.deviance df.null logLik AIC BIC deviance df.residual
## <dbl> <int> <dbl> <dbl> <dbl> <dbl> <int>
## 1 75.8 54 -28.4 60.8 64.9 56.8 53
glm(Acceptance ~ GPA + MCAT, data = MedGPA, family = binomial) %>%
glance()
## # A tibble: 1 x 7
## null.deviance df.null logLik AIC BIC deviance df.residual
## <dbl> <int> <dbl> <dbl> <dbl> <dbl> <int>
## 1 75.8 54 -27.0 60.0 66.0 54.0 52
56.83901 - 54.01419
## [1] 2.82
16 / 23

Assess

  • We want to compare a model with GPA and MCAT to one with only GPA
glm(Acceptance ~ GPA, data = MedGPA, family = binomial) %>%
glance()
## # A tibble: 1 x 7
## null.deviance df.null logLik AIC BIC deviance df.residual
## <dbl> <int> <dbl> <dbl> <dbl> <dbl> <int>
## 1 75.8 54 -28.4 60.8 64.9 56.8 53
glm(Acceptance ~ GPA + MCAT, data = MedGPA, family = binomial) %>%
glance()
## # A tibble: 1 x 7
## null.deviance df.null logLik AIC BIC deviance df.residual
## <dbl> <int> <dbl> <dbl> <dbl> <dbl> <int>
## 1 75.8 54 -27.0 60.0 66.0 54.0 52
pchisq(56.83901 - 54.01419 , df = 1, lower.tail = FALSE)
## [1] 0.0928
17 / 23

Assess

  • We want to compare a model with GPA, MCAT, and number of applications to one with only GPA
glm(Acceptance ~ GPA, data = MedGPA, family = "binomial") %>%
glance()
## # A tibble: 1 x 7
## null.deviance df.null logLik AIC BIC deviance df.residual
## <dbl> <int> <dbl> <dbl> <dbl> <dbl> <int>
## 1 75.8 54 -28.4 60.8 64.9 56.8 53
glm(Acceptance ~ GPA + MCAT + Apps, data = MedGPA, family = "binomial") %>%
glance()
## # A tibble: 1 x 7
## null.deviance df.null logLik AIC BIC deviance df.residual
## <dbl> <int> <dbl> <dbl> <dbl> <dbl> <int>
## 1 75.8 54 -26.8 61.7 69.7 53.7 51
pchisq(56.83901 - 53.68239, df = 2, lower.tail = FALSE)
## [1] 0.206
18 / 23

Use

  • We can calculate confidence intervals for the \(\beta\) coefficients: \(\hat\beta\pm z^*\times SE_{\hat\beta}\)
  • To determine whether individual explanatory variables are statistically significant, we can calculate p-values based on the \(z\)-statistic of the \(\beta\) coefficients (using the normal distribution)
19 / 23

Use

How do you interpret these \(\beta\) coefficients?

glm(Acceptance ~ MCAT + GPA, data = MedGPA, family = "binomial") %>%
tidy(conf.int = TRUE)
## # A tibble: 3 x 7
## term estimate std.error statistic p.value conf.low conf.high
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 (Intercept) -22.4 6.45 -3.47 0.000527 -36.9 -11.2
## 2 MCAT 0.165 0.103 1.59 0.111 -0.0260 0.383
## 3 GPA 4.68 1.64 2.85 0.00439 1.74 8.27
20 / 23

\(\hat\beta\) interpretation in multiple logistic regression

The coefficient for \(x\) is \(\hat\beta\) (95% CI: \(LB_\hat\beta, UB_\hat\beta\)). A one-unit increase in \(x\) yields a \(\hat\beta\) expected change in the log odds of y, holding all other variables constant.

21 / 23

\(e^{\hat\beta}\) interpretation in multiple logistic regression

The odds ratio for \(x\) is \(e^{\hat\beta}\) (95% CI: \(e^{LB_\hat\beta}, e^{UB_\hat\beta}\)). A one-unit increase in \(x\) yields a \(e^{\hat\beta}\)-fold expected change in the odds of y, holding all other variables constant.

22 / 23

Summary

Ordinary regression Logistic regression
test or interval for \(\beta\) \(t = \frac{\hat\beta}{SE_{\hat\beta}}\) \(z = \frac{\hat\beta}{SE_{\hat\beta}}\)
t-distribution z-distribution
test for nested models \(F = \frac{\Delta SSModel / p}{SSE_{full} / (n - k - 1)}\) G = \(\Delta(-2\log\mathcal{L})\)
F-distribution \(\chi^2\)-distribution
23 / 23
2 / 23
Paused

Help

Keyboard shortcuts

, , Pg Up, k Go to previous slide
, , Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow