a fine-grained but important distinction
Javert’s sense of duty and devotion is not directed at the letter of the law. It isn’t directed at some personal ideal of justice either. If it were, he would’ve broken far sooner than he did: a core belief that the letter of the law is always the highest form of justice would get utterly annihilated before long in an era of political instability and high regime turnover rate. What happens when the new regime makes laws that contradict the previous set? What happens when he witnesses an injustice or an overreach on the part of the police?
Javert’s primary belief system is at once much simpler and much more adaptable than either of those: he is devoted to authority.
I’m just going to quote Hugo here, because it’s one of the first things he tells us about Javert and his beliefs:
This man was made up of two very simple sentiments, and within reason very good ones, which he rendered almost bad by taking them to extremes: respect for authority and hatred of rebellion. And in his eyes theft, murder, any crime, was simply a form of rebellion. He looked on any state official, from the prime minister to the rural policeman, with a deep-seated blind faith. On anyone who had once crossed the legal threshold of wrong-doing he heaped scorn, loathing and disgust.
The reason Javert is able to placidly endure so many changes to the letter of the law and keep his mouth shut about whatever abuses he witnesses is because both of those things are subordinate to his guiding principle, which is: society in the form of legal authority is always in the right, and those it punishes and casts out are always in the wrong. They are unpeople, and more than that, they are a threat to respectable society.
This is why Javert doesn’t bother finding out who’s at fault for the fight between Fantine and Bamatabois. It’s not an oversight and it’s not just a bias in favor of the respectable property-holder’s version of events (which he doesn’t even bother to obtain in the book); he doesn’t give a shit about how it started, he refuses to give a shit about how it started, because that is 100% irrelevant to him. He has no desire to be ‘fair’ to a streetwalker as though there were two sides to this story. The only thing that matters to him here is the distinction between citizens and 'undesireables,’ and there is no provocation or circumstance that could excuse a whore attacking a gentleman. He doesn’t bother to find out whether Bamatabois was in the wrong because there’s no way for Fantine to have been in the right. This “us vs. them” distinction is so important to him that he will make unfair and excessive use of his discretionary powers*, refuse to give a single shit when M. Madeleine tells him Fantine wasn’t the one who started the fight, stand up to an authority figure who he thinks isn’t giving proper respect to authority, and balk and squirm about standing down when he has the letter of the law flung in his face, all in order to defend that distinction against a perceived offense.
(* Side note: Fantine is not a criminal, but she is a legal outcast. Prostitution was legal but heavily regulated; upon registering as a prostitute, Fantine would have lost most of the few civil rights women had at the time and been subject to arbitrary arrest and detention for anything the police deemed “a breach of the regulations.” Javert’s hatred is not exclusively for those who’ve broken a law, it’s for the entirety of the criminal and quasi-criminal underclasses.)
Now, this is not to say that Javert doesn't care about the letter of the law. His devotion to upholding social order is coupled with a sense of personal duty towards the rules that it’s set up to govern itself. He’s aware that his integrity is the one thing separating him from the abyss, from the “them” he hates so much, and so he is punctilious about never overstepping the boundaries that have been laid out for him. He has made irreproachability (according to the letter of the law) an essential part of his identity. But when the law gives him leeway, he uses it in the service of his guiding principle, which is “protect the 'us’, punish the 'them’ and keep them in their place.”
And his ultimate line between the two isn’t even wrongdoing, precisely–it’s the official confirmation, the mark of Cain the legal system stamps on anyone it’s deemed a transgressor. Granted, part of his job (in addition to keeping the 'scum’ in line) is to identify formerly-law-abiding people who’ve committed crimes, arrest them, and pack them off to join the ranks of the transgressors, but what his suicide note reveals is that he’s been turning a blind eye to a thousand little forms of wrongdoing and injustice when they’re committed by authority figures against the dregs of society.
So… Javert is only partly about whether the law represents justice. His ultimate devotion is not to the law itself but to his role as a dog protecting the sheep from the wolves. And his ultimate dilemma isn’t simply “wait the law might not be entirely just all the time ERROR ERROR SEGMENTATION FAULT ERROR BSOD,” it’s about whether the legal system is really a reliable way to separate the wolves from the sheep, and whether people really are irrevocably one or the other, and whether Javert’s first brush with moral duty and divine justice completely invalidates the duty towards law and society that he’s devoted his entire life to.
tl;dr people’s perception of Javert/his value system focuses too much on the act of breaking the law and not enough on the status of being a person outside the law