Screenshots from episodes 4 and 5. Things that are important here: the use of the words team and family.
Keith doesn’t want Pidge to leave the team, but he doesn’t specifically mention the team. He mentions people’s families, which is what Pidge was worried about. No doubt he was attempting to make her reconsider by using a word she was inherently interested in and was sensitive to. Others have mentioned before how ironic/nice is is of Keith to say that specifically, considering he’s an orphan and how the reason he wants Pidge to stay is because she, along with the others, are his family now.
It’s actually Shiro who uses the word team. He tells Keith that people can’t be forced into the team, they must want to stay. He’s not talking about family (specifically), but about the team of paladins they have formed.
Then cut to episode 5 and after they’ve dealt with Sendak, it’s Shiro who starts saying he can’t help but feel like Pidge was meant to be part of the team, but won’t stop her if she wants to leave. That’s when Pidge recalls her father’s words that his team was like family. Pidge is the first person to associate the two words, and Keith immediately says “welcome back to the team”, a clear call back to Shiro’s words back in episode 4 about how a team/family can’t be forced.