You haven’t done anything wrong if you used to identify as a-spec and realised later the label didn’t actually fit. You haven’t proven anybody right who told you a-spec labels weren’t real, or that your a-spec identity was only a phase. You haven’t harmed the community by identifying as a-spec and then later choosing not to.
Remember that it’s always OK to change your labels and go with whatever it is you feel fits you more and feels more comfortable to you. And that will always be the thing that matters the most.