As a person with many family members with various mental illnesses, I'm going to side with you. My cousin who has a mentat disability does not enjoy being called stupid. The same way my aunt doesn't enjoy it when her daughter is called stupid. My schizophreniac cousin doesn't enjoy being called crazy. Yeah he may make fun of it sometimes, but that doesn't mean you have the right to do so. Respecting them mean a lot more to them. They just want to fit in and laughing at them doesn't help! Sorry.
And here’s the thing about using those ableist words as insults- it says that being disabled is inherently a terrible thing, that it invalidates your thoughts and opinions to be disabled, that to call an abled person disabled is to insult them.
If you’re talking to someone and you say “oh my god, you’re just insane, stop.” What that says is that you don’t think a person with a mental illness needs to be listened to. If you call someone the R-slur, then you say that having a disability like that means that your thoughts and opinions don’t matter.
Using ableist language as an insult demands that you view the disabled as lesser.
And that’s not something that I’m cool with, on any level, at all.