The whole Eggman being betrayed thing hasn't been that prominent in a while because they broke the cycle. The only main games from Unleashed to Forces where he was betrayed was Unleashed and Lost World, but they surpassed the expectations with the latter by having him fight to take back control of his plan and still be the final boss and it was a brilliant and refreshing change from the times he was just screwed over last second. Aside from that, there was just Free Riders with him being betrayed by Metal again but that's minor.
Colors, Generations, and Forces had him as the final boss without betrayal and Lost World handled it a good way with a clever twist. And I don't think it's always so bad whenever he is betrayed, though I'd really prefer for him not to be, as long as they still give him time to shine and great moments that still show he is indeed a great threat, like the Adventure games and Lost World, for some examples. But the games in the past 12 years prove that not using him isn't required for this to change.
I'm the last person that feels like he's looked like a big joke in the modern games. The entire story tone and focus on the humor affects all of the characters, not just him, and removing him doesn't change that. It's about the tone and presentation of the stories overall, he's humorous but still evil as ever and he still has serious and threatening moments when scenes require it. If the entire story tone shifted to serious and darker again like Frontiers seems to be, the presentation of Eggman's humor and villainy would follow suit.
Forces especially is one of the games in the "meta" era where he wasn't a joke at all. It may not have been as executed as well as it could've been but he was insanely successful and smart with his tricks and backup plans. He's already been clever with them before so he had actually already learned from his mistakes prior, but seeing him acknowledge it and doing something about it actually shows intelligence. And bringing multiple more tricks from his sleeves and the grand finale was delightfully impressive.
I don't think Eggman being the final boss is ever the problem. I have no issues with him being the main villain for years in a row because he's the main villain of the series and they were already doing that well in the classic games too. And in the modern games, he's never stopped being a great threat to the planet with his great evil and schemes that cause catastrophic global destruction and fear and he's still a huge challenge for Sonic, it's not like he easily gets defeated and doesn't put up a great fight against him.
I don't have any issues with there being other villains as long as they don't have to write Eggman OOC, shit on him constantly, or betray him to make the other villain seem like a bigger threat. And I'd prefer if Eggman still gets to be involved with a good amount of significance, rather than being tossed aside or teaming up with Sonic and friends over and over. And I don't mind these other villains being the final boss as long as they don't do any of that but I'm not specifically hoping that Eggman will stop being the main villain and final boss.
I really don't think those that have any issues with the focus on humor in the later games should blame Eggman because it clearly affects the presentation of all the other characters too. Just not using him won't fix that, it can just as easily change when the entire tone of the story changes from humorous and light-hearted to serious and darker as much as it does for all the other characters too. And honestly even in a good majority of the games where he is betrayed, he isn't just treated like a joke the whole way through and still has good serious moments.
I think Eggman is one of the best and most consistently well written characters in the series, especially out of all of them in more recent games from Colors to Forces. He's still the exact same character he's always been at core, there's just been more focus on his humorous side in some games but he still has great serious and darker moments too, people just tend to overlook it because they don't like the humor focus. But it's definitely there and Eggman doesn't look like nothing but a big joke like a large portion of the fanbase treat him.
I don't think he needs to go away for a while and come back much later in order for him to not have as much focus on the humor and more focus and screentime of the evil, especially because it never actually went away entirely. I'd love for there to be more again but all it takes is the overall story tone shift and Eggman being written and presented well. I don't think that time away is what's needed for that to happen and be capable of having a great impact that reminds people that he's a serious threat that's not to be messed with.
I might sound biased with all this heavy praise and loving when he's the final boss no matter how many times but he's my favorite for a reason and many others that aren't as crazy about him tend to agree lol. I really love and want more of the focus and screentime of his evil too but I also love the humor that's just as important and I'm happy as long as I can see a good balance of them. And while maybe some games don't balance it quite as much as they should, it's still enough to make me happy and would only needs small tweaks to reach perfection.