#StackBounty: What are Regularities and Regularization?

Bounty: 50

I am hearing these words more and more as I study machine learning. In fact, some people have won Fields medal working on regularities of equations. So, I guess this is a term that carries itself from statistical physics/maths to machine learning. Naturally, a number of people I asked just couldn’t intuitively explain it.

I know that methods such as dropout help in regularization (=> they say it reduces overfitting, but I really don’t get what it is: if it only reduces overfitting, why not just call it anti-overfitting methods => there must be something more I think, hence this question).

I would be really grateful (I guess the naive ML community would be too!) if you could explain:

  1. How do you define regularity? What is regularity?

  2. Is regularization a way to ensure regularity? i.e. capturing regularities?

  3. Why do ensembling methods like dropout, normalization methods all claim to be doing regularization?

  4. Why do these (regularity/regularization) come up in machine learning?

Thanks a lot for your help.

Get this bounty!!!

Leave a Reply