Occam's Razor

Everything else being equal, choose the less complex hypothesis

It is actually state a tradeoff between fit and low complexity.

Overfitting is the major source of poor performance of machine Learning algorithms.