All right, so I’m amused by all the “Dorian is a skinny hopeless nerd without magic” headcanons, but I tend to disagree with them. This is one of the cases where I think the game models are actually closer to accurate.
I find it kind of implausible he’d have survived this long if he wasn’t in decent shape. By the time he joins the Inquisition he’s been a Tevinter drifter for a while, before camping out in the Storm Coast and Hinterlands. The vicious-mage-and-templar-infested, middle of a war zone Storm Coast and Hinterlands. With templars that can probably silence even a damn powerful Tevinter mage. He then, after having a pretty rough-scrabble time, ends up in the Inquisition where he’s fighting for his life and/or probably training most days. Even if he’d been cushioned or skinny or soft round the edges back in Tevinter, he wouldn’t be after a few months in Ferelden.
Yeah, he hates the cold and has allergies. Those things are both nice little realistic touches, and the former makes perfect sense for someone who grew up in a pseudo-Mediterranean(?) climate and is now stuck in, well, Ferelden. And when you end up in Orlais, it’s often the Emprise du Lion. Neither of those things make him weak or less able to fight.
He likes to misdirect. The whole, “look at me I’m such a useless toff, Maker forbid I get a hair out of place” thing is an act the same way “I don’t care about anyone or anything, watch me hide behind my sarcasm” is. There’s a grain of truth to it, but he’s playing it up to amuse and exasperate. Also, I find that act hilarious because he’s trying to distract from the fact he’s a tough, dangerous bloke when he needs to be, from what we see in canon. After all, half his arc is “yes, Tevinter, but not a threat, please stop glaring at me and pointing swords at me.” Of course he’s playing down the fact he’s a brutal, capable fighter with plenty of ingenuity.
That said, Mr "What do you mean robes need shoulders and sleeves?” seems to like to show off. He’d probably take pride in being in decent shape, the same way he seems to take pride in most other aspects of his appearance. Exhibit A: mate, why do your more advanced robes have less sleeve, this makes no sense, I know it’s video-game logic, but honestly… As said: Emprise. Du. Lion. It also makes sense for the whole “perfect body, perfect mind” and “partners only valued him for physical attraction” ideas. That kind of showing off is something he knows and is comfortable with.
The first time he’s seen in In Hushed Whispers, he’s literally beating a demon back, no magic involved. And succeeding. I tend to theorise that, unlike southern Circles that want to keep mages as weak as possible, there’s at least some combat training in Tevinter. After all, they are in the middle of an ongoing war with Seheron. And DA2, with things like staff blades, has laid a precedent for melee mages. I mean… why not?
Basically, Dorian Pavus is a hot-headed, hard-drinking, strong as hell tough bastard who could probably take you in a bar fight but would much prefer to talk his way out of it instead, and like all the Inquisition mages, is just as much a fighter as anyone else. And that’s my headcanon.