It’s possible to match a data set optimally with one parameter.
Model: y=sin(bx), where y is scaled such that all values fall between 0 and 1 exclusive. The difficulty of hitting every point rises with the number of data points, but that just means you need bigger values of b. The “model” will look like an almost fully filled space with a sin curve oscillating so fast it looks like a series of vertical lines. Yet it hits every single point (because in a large enough option space, I can do that) when possible or the exact midpoint when not. Plausibly 100% perfection is impossible in many cases, but a sufficiently close approximation probably is.
If this understanding of overfitting sin waves is correct, doesn’t that suggest a flaw in how we penalize complexity in model-fitting?
I’ll invest some money on an exercise or fitness program the day I see one represented with people from all sizes, sure it’s a fitness program so you want your super fit boy and big butt bigger arms kind of girls, but give me a chubby girl who gives her 100% on the promo for the fitness program, or an average body guy who struggles with the reps of the arm day but manages to finish it.
I understand how a fitness brand needs “fit” bodies for promotion, but nah son, the industry is full of those perfect body “goals”, and I don’t want a “chubby” person as the ~before~but as one of the main faces of the brand, showing that fitness is for everyone, that hard work doesn’t comes only from perfect fit models.