Regression Analysis

Author

Steven E. Finkel
📧 finkel@pitt.edu

Course Description

This course is the next helping of statistics and research methodology after PS2020. We will begin with a thorough investigation of regression and the general linear model, and then move on to more advanced techniques such as logit and probit analysis, models for ordinal dependent variables, and models for causal inference. Students will be exposed to the theory of these methods as well as to practical ways of using them in concrete research situations.

There are two main goals of the course: to enable students to read, understand and critique existing quantitative literature in political science; and to have students acquire the skills to conduct original quantitative research in their own substantive field of interest.

Texts

  • Gujarati, Damodar, and Dawn Porter. 2009. Basic Econometrics, 6th Edition. McGraw-Hill Irwin.
  • Long, J. Scott. 1997. Regression Models for Categorical and Limited Dependent Variables. Sage Publications, Inc.
  • Angrist, Joshua, and Jörn-Steffen Pischke. 2009. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press.

Supplemental References

  • Long, J. Scott, and Jeremy Freese. 2014. Regression Models for Categorical Dependent Variables Using Stata, 3rd Edition. Stata Press.
  • Angrist, Joshua, and Jörn-Steffen Pischke. 2015. Mastering ’Metrics. Princeton University Press.
  • Mitchell, Michael. 2021. Interpreting and Visualizing Regression Models Using Stata, 2nd Edition. Stata Press.
  • Wooldridge, Jeffrey. 2015. Introduction to Econometrics: A Modern Approach, 6th Edition. South-Western Cengage Learning.

Course Requirements

Grades will be based on a 20–25 page research paper (46%, including 10% for preparation of a poster presentation of the paper), and three homework exercises which relate to specific statistical methods and problems we will discuss (54% altogether). The paper will be a quantitative analysis, using multiple regression or some of the other methods we cover in the course, of data that you will collect or access from social science archives or other sources. Ideally, the paper should have some substantive interest to you or be relevant to your studies in the graduate program. The paper will discuss your basic theoretical framework, your hypotheses, statistical models, results, possible problems with the analysis, and what you may have done to correct or account for these problems. It will conclude with a discussion of the relevance of your findings for the general topic and for future research. The homework exercises will be periodic problems or data sets to analyze and will illustrate aspects of the statistical techniques being covered in class.

Course Outline

The course is organized by units and then topics within units. We will maintain a certain amount of flexibility with the schedule, so that we can spend more time on some topics/units and scale back on others as circumstances warrant.

Unit 1: Fundamentals of Linear Regression

1. Bivariate Regression

  • Gujarati, Damodar N., and Dawn C. Porter. 2009. Basic Econometrics. New York: McGraw-Hill Education, Introduction, Chapters 1–2, and Chapter 3: pp. 55–61 and 73–80. 
  • Long, J. Scott, and Jeremy Freese. 2014. Regression Models for Categorical Dependent Variables Using Stata. College Station, TX: Stata Press, Chapter 2. (Optional)
  • Mitchell, Michael. 2021. Interpreting and Visualizing Regression Models Using Stata, 2nd Edition. Stata Press, Chapters 1–2, Sections 2.1–2.2. (Optional)
  • Wooldridge, Jeffrey M. 2015. Introductory Econometrics: A Modern Approach. Boston, MA: Cengage Learning, Chapter 2. (Optional)

2. Assumptions of Ordinary Least Squares

  • Gujarati, Damodar N., and Dawn C. Porter. 2009. Basic Econometrics. New York: McGraw-Hill Education, Chapter 3: pp. 61-73 and 92–96 and Chapter 4: pp. 97-102.
  • Long, J. Scott, and Jeremy Freese. 2014. Regression Models for Categorical Dependent Variables Using Stata. College Station, TX: Stata Press, Chapter 2. (Optional)
  • Mitchell, Michael. 2021. Interpreting and Visualizing Regression Models Using Stata, 2nd Edition. Stata Press, Chapters 1–2, Sections 2.1–2.2. (Optional)
  • Wooldridge, Jeffrey M. 2015. Introductory Econometrics: A Modern Approach. Boston, MA: Cengage Learning, Chapter 2. (Optional)

3. Hypothesis Testing

  • Gujarati, Damodar N., and Dawn C. Porter. 2009. Basic Econometrics. New York: McGraw-Hill Education, Introduction, Chapter 5.
  • Long, J. Scott, and Jeremy Freese. 2014. Regression Models for Categorical Dependent Variables Using Stata. College Station, TX: Stata Press, Chapter 2. (Optional)
  • Mitchell, Michael. 2021. Interpreting and Visualizing Regression Models Using Stata, 2nd Edition. Stata Press, Chapters 1–2, Sections 2.1–2.2. (Optional)
  • Wooldridge, Jeffrey M. 2015. Introductory Econometrics: A Modern Approach. Boston, MA: Cengage Learning, Chapter 2. (Optional)

4. Multiple Regression

  • Gujarati, Damodar N., and Dawn C. Porter. 2009. Basic Econometrics. New York: McGraw-Hill Education, Chapter 5: pp. 129–133, Chapter 6: pp. 157–159, and Chapters 7–8. 
  • Wooldridge, Jeffrey M. 2015. Introductory Econometrics: A Modern Approach. Boston, MA: Cengage Learning, Chapters 3–5. 
  • Mitchell, Michael. 2021. Interpreting and Visualizing Regression Models Using Stata, 2nd Edition. Stata Press, Chapter 2, Section 2.3.

5. Dummy Variable Regression

  • Gujarati, Damodar N., and Dawn C. Porter. 2009. Basic Econometrics. New York: McGraw-Hill Education, Chapter 9. 
  • Mitchell, Michael. 2021. Interpreting and Visualizing Regression Models Using Stata, 2nd Edition. Stata Press, Chapters 7 and 10. 
  • Nguyen, Trang Quynh, Ian Schmid, and Elizabeth A Stuart. 2020. “Clarifying Causal Mediation Analysis for the Applied Researcher: Defining Effects Based on What We Want to Learn.” Psychological Methods 26(2): 255–271. 
  • Wooldridge, Jeffrey M. 2015. Introductory Econometrics: A Modern Approach. Boston, MA: Cengage Learning, Chapter 7 to p. 223. (Optional) 
  • Mitchell, Michael. 2021. Interpreting and Visualizing Regression Models Using Stata, 2nd Edition. Stata Press, Chapter 8. (Optional) 
  • StataCorp. 2023. Stata Causal Inference and Treatment-Effects Estimation Reference Manual, Release 18, pp. 188–219. (Optional)

Unit 2: Regression Models: Extensions and Problems

1. Functional Form: Non-Linear and Non-Additive Models

  • Gujarati, Damodar N., and Dawn C. Porter. 2009. Basic Econometrics. New York: McGraw-Hill Education, Chapter 6: pp. 159–175, and Chapter 13: 470–482, 486–498.
  • Brambor, Thomas, William R. Clark, and Matt Golder. 2006. “Understanding Interaction Models: Improving Empirical Analyses.” Political Analysis 14: 63–82. 
  • Mitchell, Michael. 2021. Interpreting and Visualizing Regression Models Using Stata, 2nd Edition. Stata Press, Chapters 3 and 5. 
  • Wooldridge, Jeffrey M. 2015. Introductory Econometrics: A Modern Approach. Boston, MA: Cengage Learning, Chapter 6. (Optional) 
  • Mitchell, Michael. 2021. Interpreting and Visualizing Regression Models Using Stata, 2nd Edition. Stata Press, Chapter 11. (Optional)

2. Violations of Assumptions: Multicollinearity, Heteroskedasticity, Autocorrelation

  • Gujarati, Damodar N., and Dawn C. Porter. 2009. Basic Econometrics. New York: McGraw-Hill Education, Chapters 10–12. 
  • Wooldridge, Jeffrey M. 2015. Introductory Econometrics: A Modern Approach. Boston, MA: Cengage Learning, Chapters 8, 10–11. (Optional)

3. Endogenous Regressors

  • Gujarati, Damodar N., and Dawn C. Porter. 2009. Basic Econometrics. New York: McGraw-Hill Education, Chapter 13: pp. 482–486. 
  • Wooldridge, Jeffrey M. 2015. Introductory Econometrics: A Modern Approach. Boston, MA: Cengage Learning, Chapter 15. 
  • Wooldridge, Jeffrey M. 2015. Introductory Econometrics: A Modern Approach. Boston, MA: Cengage Learning, Chapter 16. (Optional) 
  • Maydeu-Olivares, Albert, Dexin Shi, and Amanda J. Fairchild. 2020. “Estimating Causal Effects in Linear Regression Models With Observational Data: The Instrumental Variables Regression Model.” Psychological Methods 25(5): 243-258.

Unit 3: Models for Non-Continuous Dependent Variables

1. Introduction to Logit and Probit Models

  • Long, J. Scott. 1997. Regression Models for Categorical and Limited Dependent Variables. Thousand Oaks, CA: Sage Publications, Inc., Chapters 1 and 3 to p. 52. 
  • Mitchell, Michael. 2021. Interpreting and Visualizing Regression Models Using Stata, 2nd Edition. Stata Press, Chapter 18, Sections 18.1–18.2.

2. Estimation and Interpretation of Logit and Probit Models

  • Long, J. Scott. 1997. Regression Models for Categorical and Limited Dependent Variables. Thousand Oaks, CA: Sage Publications, Inc., Chapter 2 (especially pp. 25–33), pp. 52–end, and Chapter 4. 
  • Hanmer, Michael J., and Kerem Ozan Kalkan. 2013. “Behind the Curve: Clarifying the Best Approach to Calculating Predicted Probabilities and Marginal Effects from Limited Dependent Variable Models.” American Journal of Political Science 57(1): 263–277. 
  • Long, J. Scott, and Jeremy Freese. 2014. Regression Models for Categorical Dependent Variables Using Stata. College Station, TX: Stata Press, Chapters 3–6. (Optional)

3. Models for Ordinal and Nominal Outcomes

  • Long, J. Scott. 1997. Regression Models for Categorical and Limited Dependent Variables. Thousand Oaks, CA: Sage Publications, Inc., Chapters 5–6. 
  • Mitchell, Michael. 2021. Interpreting and Visualizing Regression Models Using Stata, 2nd Edition. Stata Press, Chapter 18, Sections 18.3–18.4. 
  • Long, J. Scott, and Jeremy Freese. 2014. Regression Models for Categorical Dependent Variables Using Stata. College Station, TX: Stata Press, Chapters 7–8. (Optional)

Unit 4: Models for Causal Inference

1. Counterfactuals, Potential Outcomes, and Causal Inference

  • Angrist, Joshua D., and Jörn-Steffen Pischke. 2009. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press, Chapters 1–2.

2. Selection on Observables: Regression and Matching

  • Angrist, Joshua D., and Jörn-Steffen Pischke. 2009. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press, Chapter 3. 
  • Gangl, Thomas. 2015. “Matching Estimators for Treatment Effects.” In The SAGE Handbook of Regression Analysis and Causal Inference, edited by Henning Best and Christof Wolf, London: Sage Publications, Inc., pp. 251–276. 
  • Morgan, Stephen L., and Christopher Winship. 2007. Counterfactuals and Causal Inference: Methods and Principles for Social Research. Cambridge University Press, Chapters 3–4. (Optional)

3. Selection on Unobservables: Instrumental Variables and Difference-in-Differences

  • Angrist, Joshua D., and Jörn-Steffen Pischke. 2009. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press, Chapters 4–5. 
  • Sovey, Allison J., and Donald P. Green. 2011. “Instrumental Variables Estimation in Political Science: A Reader’s Guide.” American Journal of Political Science 55(1): 188–200. (Optional) 
  • Muller, Christopher, Christopher Winship, and Stephen L. Morgan. 2015. “Instrumental Variables Regression.” In The SAGE Handbook of Regression Analysis and Causal Inference, edited by Henning Best and Christof Wolf, London: Sage Publications, Inc., pp. 277–299. (Optional)

4. Fixed Effects and Panel Models

  • Allison, Paul D. 2009. Fixed Effects Regression Models. Thousand Oaks, CA: Sage Publications, Inc., Chapter 2. 
  • Bell, Andrew, and Kelvyn Jones. 2015. “Explaining Fixed Effects: Random Effects Modeling of Time Series Cross-Section and Panel Data.” Political Science Research and Methods 3(1): 133–153.