My two cents on the first ‘you’
Alright so I have been thinking about the thing and I have concluded that I think that “is the first you singular or plural” is pretty much a particularly fruitful way to approach the scene. Hear me out.
Who is Cas referring to up to this moment? The three of them. After saying “you’re my family” he turns specifically to Mary and nods, like he’s telling her “yes, you too”. So far, “you” means “Dean, Sam, and Mary”.
Why would he suddenly shift from a “you” that means “Dean, Sam, and Mary” to a “you” that means something different? He’s making a speech and it makes no sense for him to change who he’s addressing to with no indication he’s doing so. I think his addressee is the same - the three of them.
If the speech ended here, if he said the thing about not wanting to see them die now, the scene would be Cas tells Dean, Sam and Mary that he loves them. Period. Dean wouldn’t stand out particularly - yes, the camera cuts to him immediately and the direction of Cas’ eyes and everything, but Cas’ words themselves wouldn’t give you reasons to think that there’s something in them that hints at specific feelings he has for Dean.
Cas feels the need to clarify what he meant with that “you”. Which would be absolutely unnecessary and make no sense unless you assume that Cas loves Dean in a way that differs from his feelings for Sam and Mary.
He’s been talking to his family, to the three of them, why would he feel the need to clarify that he means the three of them?
Because there is a chance that they might have assumed he was referring to Dean. There is a chance that Sam and Mary would think he was talking to Dean only, not to them. Because everyone in that room knows that Cas’ feelings for Dean are not the same as Cas’ feelings for Sam and Mary, so Cas is saying “I am expressing my love for all of you”.
Sam’s face here could either mean (or mean a mixture of both, more likely) “oh, you mean you love me too” or “oh you love Dean so much and it’s so obvious I thought you were only talking about him”. Kinda.
Basically Cas says I love you. And by you I don’t just mean Dean which I know each of us knows I love because literally every pebble outside knows that because the entire universe knows, in fact it’s so obvious you’re probably thinking that’s what I meant with ‘I love you’ right, so what I’m saying is that I mean that I care deeply about all of you because you all are my family and you all have been important to me and made me into the person I am now.
Basically the point is that everything he’s said before the “I love all of you” can be interpreted in two different ways - i.e. applied to Dean only (and it gets certain connotations) or applied to the three of them (and it gets different connotations). And the fact that Cas clarifies it, means that he’s aware of that possible dualism and he’s aware that everyone in the room is aware of it.
And if Perez is making the characters acknowledge that dualism (i.e Cas’ feelings for Dean being different than Cas’ feelings for the others), it means that that dualism is real.
Basically the scene isn’t telling us the nature of Cas’ feelings for Dean. That needs to happen still. It’s a little sign telling us that they’re different than Cas’ feelings for Sam or Mary.
In 11x23 Cas made a pained face when Dean told him that he was their brother. That’s what we’re exploring right now: is it possible to use the same label for Cas’ and Dean’s relationship as for Cas’ and Sam’s relationship? It’s pretty much what Robbie Thompson foreshadowed at with the line “what about Sastiel” - in the context of the episode it was a joke, but
nothing is just a joke from a meta point of view it means, what’s the difference between ‘Dean and Cas’ and ‘Sam and Cas’?
Basically, in order to make Destiel shift from subtext to text, you have to answer the question ‘what about Sastiel’ first, i.e. elaborate on the difference between Cas’ feelings for Sam and Cas’ feelings for Dean. And Cas’ speech in 12x12 of course doesn’t mean all of this stuff textually, but, extradiegetically, it’s a sign that the show is setting the ground for the answers to that kind of questions.