I dunno, the more I watch him push Jesse around because in his view Jesse's just a "loser" the more I come to think that actually, while he was always probably decent enough to people who he viewed as his social equal, he was also probably always filled with unshakeable contempt for people he deemed to be beneath him. So a junkie's always a junkie and is always going to be a junkie to him, and he just can't allow the possibility of any kind of redeeming features in any such people.
Which kind of makes him an arsehole, really. It's just that until recently he was never desperate enough for the arseholishness to manifest itself too clearly.