An excellent practical introduction to all things regression-related. Harrell’s book will not tell you about the Gauss-Markov assumptions and the asymptotic properties of estimators. Instead, it is an opinionated guide to the realities of statistical modelling.

The introduction chapter addresses important meta-questions about the importance of models and things to consider before modelling. Chapter 2 gives a good overview of how to interpret and evaluate regression models, while providing a comparison with other model formulations like decision trees and ML. Chapter 4 provides several strategies for multivariate modelling; Chapter 5 suggests some very interesting techniques for validating models.

I derived a lot of value from the book. It contains a large toolbox of statistical techniques and it’s full of rules of thumb that academic statisticians are often loath to provide: for example, you need at least 15 data points per predictor.

I skimmed over some parts of the book as they aren’t relevant to me at the moment: survival analysis, logistic regression, longitudinal response. But I’m comfortable in the knowledge that I can flick back to those chapters when and if.

This is a selection of advice that resonates with me:

• Prefer regression over classification. It allows you to separate the decision function from the actual model.
• Random methods:
  • Add noise to y variables before doing feature selection to avoid overfitting.
  • Apply your pipeline to the same dataset with shuffled y (no signal). If you still get a fit, you have overfit.
• Be careful “spending” degrees of freedom by e.g. looking at scatter plots before regressing.
• Avoid stepwise feature selection