21

I am looking for blogs that focus on the mathematical theory of Statistics and Machine Learning, ideally at a research or "advanced" level.

The blog doesn't have to be solely about these topics but ideally most of the posts would be exposing the mathematical theory of an idea/concept/algorithm that is either directly or closely related to them. Here is an example of what I'm looking for, and I will add another one as an answer (Disclaimer : I have no affiliation with any of the authors of the blogs I link):

  • Gregory Gundersen's blog neatly presents the theory of many well-known (and some lesser-known) algorithms and results in Statistics, such as Conjugate Gradient Descent, Ordinary Least Squares, Hidden Markov Models... Some of the posts contain illustration with available source code

Please limit your answer to less than 2-3 links and provide a short description for each blog, as above.

Stratos supports the strike
  • 1,628
  • 2
    See previous and continuing discussion on https://stats.meta.stackexchange.com/questions/6385/why-are-these-blog-recommendations-questions-considered-off-topic While I share the misgivings of several members about these threads, I don't regard them as utterly off-topic. – Nick Cox Sep 21 '22 at 06:45
  • 3
    Maybe https://errorstatistics.com could be an answer? A blog by Deborah G. Mayo who writes about philosophical foundations of statistics (like the controversy between frequentist and Bayesian approaches). It is not the hardcore mathematics behind statistics, but is is about foundations and underlying principles behind the application of statistical methods. – Sextus Empiricus Sep 21 '22 at 13:06
  • 1
    Math3ma is perhaps closer to pure math than Stats or ML, but they have articles that connect to these subjects at an advanced level. Not quite an answer, but worth a mention. – Galen Sep 21 '22 at 16:12

8 Answers8

9

Andrew Gelman: https://statmodeling.stat.columbia.edu. Gelman is a professor of statistics and political science at Columbia, and has co-authored several statistics books, including Bayesian Data Analysis and Regression and Other Stories. I strongly disagree with most of his politics, but his statistics is generally sound.

Adrian Keister
  • 5,862
  • 5
    I don't regard this blog, which I used to follow, as focused on 'the mathematical theory of Statistics and Machine Learning, ideally at a research or "advanced" level'. – Nick Cox Sep 21 '22 at 06:41
  • @NickCox isn't this blog about "the mathematical theory of an idea/concept/algorithm that is either directly or closely related to them.", where we can fill in 'Bayesian statistics' in place of the concept? – Sextus Empiricus Sep 21 '22 at 10:28
  • I remember one interaction between that blog and a question here : https://stats.stackexchange.com/questions/389287/how-to-map-an-estimated-risk-ratio-of-0-76-to-increased-life-expectancy or is that not theoretic enough? – Sextus Empiricus Sep 21 '22 at 10:32
  • I am reacting to "focus on" in the question; if you want to take that as meaning "sometimes mention", you are right, but then we are disagreeing about English usage. I would certainly recommend that anyone with a strong statistical side looks at Gelman's blog and works out how far it is interesting or useful to them. I admire Gelman's evident intelligence and sheer energy, but found that there was far too much material for me to follow indefinitely (and like my own very different posts, he tends to repeat personal preoccupations). Still, nothing stops sampling! – Nick Cox Sep 21 '22 at 10:42
8

Francis Bach's Machine Learning Research blog is an "easy to digest" introduction to some of his research works and related topics ("easy" as in easier than reading the original papers).

It contains many excellent in-depth writings about kernel methods, optimization algorithms, linear algebra and highlights how these topics interact with each other as well as their applications in Machine Learning/Statistical Learning Theory.

Stratos supports the strike
  • 1,628
4

https://statisticaloddsandends.wordpress.com/ reminds me of Gunderson blog, nicely written with code and clear explanations.

Kozolovska
  • 1,355
3

ICLR recently introduced its Blog Track and its taken inspiration from some blogs like Bach's. Best thing is that it's peer-reviewed and contains diverse topics from diverse authors (often a group of authors).

Failed Scientist
  • 119
2

In the last couple of years I have warmed up to using geometry to understand deep learning models, and indeed various types of statistical models. While I recommend the book Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges, you can also find a list of blogs related to the topic.

Galen
  • 8,442
  • 3
    The alliteration is good to see, given also your identifier. One interest of mine is circular data, which I explain to be about data on the compass, clock or calendar, a curious coincidence. – Nick Cox Sep 21 '22 at 06:50
2

An Outsider's Tour of Reinforcement Learning by Ben Recht gives a short introduction into RL and draws connection to control theory.

Alex bGoode
  • 101
1

This is neither really a blog nor just about statistics and many times very basic, but I found many good advices and ideas in there so I decided to add it as an answer

https://chrisalbon.com/#code_statistics

Eulenfuchswiesel
  • 121
0

Towards data science a collection of articles focussing on data science, machine learning, artificial intelligence and programming. It is written by various authors. The articles often focus on explaining some technique or area.

A quick search finds some links on the website here but possibly there are more indirect links.

Sextus Empiricus
  • 77,915