There was never supposed to be a Golden Age
“It’s your destiny Merlin!” has been oft-repeated throughout the show. Kilgarrah tends to be a prime motivator, always adding new aspects to his destiny. However the core is still the same: bring about a magic-friendly, united Albion.
But at the end, what did he tell Merlin? That his destiny was completed, that he’d done his part so that Arthur would rise again when the Isles needed him the most, despite Merlin being built up for this destiny all throughout seasons 1-4. I can only conclude that his destiny was to prevent the Golden Age. That Merlin was supposed to choose Mordred’s death in the Dsir. That Arthur was supposed to die, and that Merlin was supposed to wait for him for millennia. That Guinevere was supposed to take the throne and Mordred was supposed to destroy Camelot. That the Saxons were supposed to invade post-Arthur and wipe out Celtic England.
It was all supposed to happen.
Not to mention, in the Dsir, the entire episode arc basically shows how Merlin’s being manipulated into picking a certain side. The “mouthpiece of the Goddess” lied to him. They knew that his entire decision hinged on Arthur’s safety, and he had to kill Mordred to ensure it. So he agrees to the choice that he thinks will eliminate Mordred. But wait, Mordred isn’t dead.
Basically, Kilgarrah, the Triple Goddess and the Dsir were working together to prevent the golden age.
And I sound like a conspiracy nut.