Skip to content

Regression

Introduction

The idea of looking at a lot of data samples and trying to predict the dependent variable in a continuous numeric domain is called regression in statistical terms.

Assumptions

In order to perform regression on any dataset, it must satisfy the following assumptions:

  1. Normality: The erros are assumed to be normally distributed
  2. Independent: The errors must be independent of each other
  3. Mean and Variance: They must have zero mean and constant variance (this property of having a constant variance is also called homoscedasticity

These assumptions are usually verified using Q-Q Plots, S-W test etc.

This chapter offers introduction to various kind of regressions and their use cases.

  1. Linear Regression
  2. Logistic Regression
  3. Polynomial Regression
  4. Stepwise Regression
  5. Ridge Regression
  6. Lasso Regression
  7. ElasticNet Regression
  8. Support Vector Regression
  9. Decision Tree Regression
  10. Random Forest Regression
  11. Gradient Boosting & AdaBoost
  12. XGBoost Regression
  13. Bayesian Linear Regression
  14. Generalized Linear Model (GLM)
  15. Poisson Regression
  16. Negative Binomial Regression
  17. Cox Regression
  18. Multivariate Adaptive Regression Splines (MARS)
  19. Robust Regression
  20. Principal Components Regression (PCR)
  21. Partial Least Squares (PLS) Regression
  22. Tweedie Regression
  23. Quantile Regression
  24. Neural Network Regression
  25. SVR (Support Vector Regression)
  26. Stochastic Gradient Descent Regression
  27. k-Nearest Neighbors Regression
  28. LightGBM Regression
  29. CatBoost Regression

ARTICLE REFERENCES

  1. Q-Q Plot