compare bayesian linear regression VS linear regression

Question

Recently I learnt the bayesian linear regression model, but what I'm confused is that in which situation we should use the linear regression, and when to use the bayesian version. How about the performance of these two? And is the bayesian logistic regression and logistic regression the same? I read a paper about using bayesian probit regression to predict ads CTR, I just wonder why using bayesian version?

Answer 1

In your two cases, linear regression and logistic regression, the Bayesian version uses the statistical analysis within the context of Bayesian inference, e.g., Bayesian linear regression.

Per wikipedia,

This (ordinary linear regression) is a frequentist approach, and it assumes that there are enough measurements to say something meaningful. In the Bayesian approach, the data are supplemented with additional information in the form of a prior probability distribution. The prior belief about the parameters is combined with the data's likelihood function according to Bayes theorem to yield the posterior belief about the parameters.

The usual way of Bayesian analysis (adding the Bayesian taste):

1. Figure out the likelihood function of the data.
2. Choose a prior distribution over all unknown parameters.
3. Use Bayes theorem to find the posterior distribution over all parameters.

Why Bayesian version? ^[1]

• Bayesian models more flexible, handles more complex models.
• Bayesian model selection probably superior (BIC/AIC).
• Bayesian hierarchical models easier to extend to many levels.
• Philosophical differences (compared to frequentist analysis).
• Bayesian analysis more accurate in small samples (but then may depend on priors).
• Bayesian models can incorporate prior information

This hosts some good lecture slides about Bayesian analysis.