yea, I think some people just…fundamentally misunderstand the tone and function of the show since its entirely built around small character moments. Those small character moments add to the character’s personal progression and allow us to connect with the characters so we actually care when the big plot stuff happens. A big flaw in a lot of action shows is that they only develop the characters enough to drive the plot, usually only using them for big character moments and otherwise just giving them a gimmick as false characterization (this character always cracks jokes, this character is always checking themselves out in the mirror), so you don’t end up caring about what happens to the characters because they’re so underdeveloped.
To have a character driven show, you need small character moments that humanize the character. Through this, you can develop a character overtime and give them development in a natural way, rather than just introducing character elements only when they’re relevant and then ‘solving’ that character element within the same episode.
Besides, “filler” doesn’t mean “not a major advancement on the plot”, it means an inconsequential episode that can be completely removed without a negative impact on story comprehension, because its essentially sawdust or the air in a bag of chips - it just fills space. If you remove the episodes of SU people consider “filler”, you’re going to end up with a garbled mess with horrendous pacing, confusing character actions, plot points that seem to come out of nowhere, and no real sense of urgency or drama because we’d have zero connection to any of the characters and thus not really care about whether or not things turn out well for them