This is how I, too, interpret the Bashir-Garak dynamic. Both men have traumatic childhoods that were the outcome of manipulative parents, but what allows Bashir to salvage his sense of love if that, in a twisted sense, his parents really did love him. Garak, however, was used as a means to an end by his father, and his mother was too low on the Cardassian social structure to do much about it while likely also accepting the social convention.
Garak is capable of love, but that sense was atrophied on purpose to protect himself, for the sake of his position in the Obsidian Order, and in Cardassian society as a whole. Love is latent within him during his time at DS9; he is able to recognize love but discards it in favor of forming alliances. This is why he approaches Bashir. Here is a pretty toy that he covets, and the doctor’s position in Starfleet will allow Garak to keep his eye on the Federation. (Doctors exist both within and without the chain of command, allowing Bashir—and thus Garak—greater potential leeway for getting/giving information.) But love does not enter Garak’s affections (for he is affectionate and flirts in a Cardassian manner), but if he feels inklings of love he must quash them to the level of unconscious yearning.
I feel that Garak’s love for his father, Enabran Tain, exists in this same repressed space. True, Garak openly looks for his father’s approval (torturing Odo, skirting back into Cardassian society when tempted but not quite giving in, etc.) in a way that is acceptable to the son of the head of the Obsidian Order. But approval and love are not the same thing. The only time we see true love for his father emerging, and a little boy’s craving for love that never came to be, is when they are in the Dominion prison (”By Inferno’s Light”). Garak’s shell breaks precisely once, when he lifts his head and sighs, obviously crying, as he asks for Tain to acknowledge him as his son. This is a moment that leaves Bashir in clear distress as he looks on. In the end, Garak never gets his father’s love, but he does get his father’s approval (”I was proud of you that day.”) and that is close enough for Garak.
Had a loving, romantic relationship between men been allowed to happen on 1990s television, this moment in the Dominion prison would have been a perfect turning point for Bashir and Garak to openly fall in love from that season on out. I would argue that it is at that moment that Garak begins to lose the armor of being “A Cardassian and Son of Tain” and instead becomes a man who is capable of love, even if he does not recognize it at first. He allows Bashir to witness him at his most vulnerable, in excruciating emotional pain, something that a secret government operative like Garak would not have been able to allow at any earlier time in his life. Cardassians as a whole do not allow people outside their family to witness family rites, but here is Garak, allowing Bashir into the circle of family, as one might a spouse. There is a connection deep between Garak and Bashir, even if Garak cannot yet give voice to its exact shape.